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Preface 


For  the  thirty-second  year,  the  Research  and  Theory  Division  of  the  Association  for  Educational  Communications 
and  Technology  (AECT)  is  sponsoring  the  publication  of  these  Proceedings.  Papers  published  in  this  volume  were 
presented  at  the  national  AECT  Convention  in  Anaheim,  CA.  A limited  quantity  of  these  Proceedings  were  printed 
and  sold  in  both  hardcopy  and  electronic  versions.  Volume  #1  is  available  through  the  Educational  Resources 
Clearinghouse  (ERIC)  System.  Proceedings  volumes  are  available  to  members  at  AECT.ORG. 

The  Proceedings  of  AECT’s  Convention  are  published  in  two  volumes.  Volume  #1  contains  papers  dealing 
primarily  with  research  and  development  topics.  Papers  dealing  with  the  practice  of  instructional  technology 
including  instruction  and  training  issues  are  contained  in  Volume  #2. 

REFEREEING  PROCESS:  Papers  selected  for  presentation  at  the  AECT  Convention  and  included  in  these 
Proceedings  were  subjected  to  a reviewing  process.  All  references  to  authorship  were  removed  from  proposals 
before  they  were  submitted  to  referees  for  review.  Approximately  sixty  percent  of  the  manuscripts  submitted  for 
consideration  were  selected  for  presentation  at  the  convention  and  for  publication  in  these  Proceedings.  The  papers 
contained  in  this  document  represent  some  of  the  most  current  thinking  in  educational  communications  and 
technology. 
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Teaching  with  Web  2.0  Technologies:  Benefits,  Barriers  and  Best  Practices 


Yun-Jo  An 
Bosede  Aworuwa 
Glenda  Ballard  & 

Kevin  Williams 

College  of  Liberal  Arts  and  Education 
Texas  A&M  University-Texarkana 

Abstract 

The  purpose  of  this  study  was  to  explore  best  practices  in  teaching  with  Web  2.0  technologies  as  well  as 
the  benefits  and  barriers  associated  with  the  use  of  Web  2.0.  Participants  in  this  study  were  14  university  instructors 
who  had  considerable  experience  in  teaching  with  Web  2.0  technologies.  A Web-based  survey  was  used  to  collect 
data  for  this  study.  The  study  results  indicate  that  the  major  benefits  of  using  Web  2.0  technologies  in  teaching 
include  (1)  interaction,  communication  and  collaboration,  (2)  knowledge  creation,  (3)  ease  of  use  and  flexibility,  and 
(4)  writing  and  technology  skills.  The  major  barriers  university  instructors  encounter  in  teaching  with  Web  2.0 
technologies  include  (1)  uneasiness  with  openness,  (2)  technical  problems,  and  (3)  time.  The  survey  results  also 
provided  insightful  guidelines  and  tips  for  teaching  with  Web  2.0  technologies. 

Descriptors:  Web  2.0;  Benefits  and  Barriers;  Best  Practices 

Introduction 

Web  1.0  was  read-only  where  Internet  users  went  online  to  find  information.  It  was  similar  to  going  to  the 
library  to  find  books.  With  Web  2.0,  which  is  read/write,  people  have  become  active  participants  and  content 
creators.  They  not  only  find  information  on  the  Internet,  but  they  also  create  and  share  content  (Thompson,  2007). 
According  to  Edison  survey  (2009),  about  15  percent  of  podcast  consumers  are  also  content  creators  compared  to 
five  percent  of  those  who  are  not  podcast  consumers.  Web  2.0  technologies  - such  as  blogs,  wikis,  podcasting, 
social  bookmarking,  and  social  networking  sites  - have  allowed  users  to  easily  publish  content  online  and  to  connect 
and  network  with  other  people  from  all  over  the  world  who  have  similar  interests.  The  use  of  tags  particularly 
enables  us  to  collectively  categorize  and  find  content  easily.  In  a nutshell,  Web  2.0  could  be  characterized  by 
openness,  user  participation,  knowledge  sharing,  social  networking  and  collaboration,  user-created  content,  and 
folksonomy  (Alexander,  2006;  Brown  & Adler,  2008;  Downes,  2005;  Thompson,  2007;  Richardson,  2009). 

Web  2.0  in  Teaching  and  Learning 

As  addressed  above,  Web  2.0  technologies  have  “blurred  the  line  between  producers  and  consumers  of 
content  and  has  shifted  attention  from  access  to  information  toward  access  to  other  people”  (Brown  & Adler,  2008, 
p.  18).  Emphasizing  a participatory  culture,  Web  2.0  technologies  encourage  and  enable  teachers  and  learners  to 
share  ideas  and  collaborate  in  innovative  ways.  They  also  force  educators  to  rethink  the  way  we  teach  and  leam  and 
to  transform  our  education  practices  so  that  we  can  support  more  active  and  meaningful  learning  that  involves 
“learning  to  be”  as  well  as  “learning  about.” 

Web  2.0  has  the  potential  to  create  more  interactive  and  powerful  learning  environments  in  which  learners 
become  knowledge  creators,  producers,  editors,  and  evaluators  (Richardson,  2009).  Learners’  critical  thinking  skills 
can  be  enhanced  through  the  opportunity  to  regularly  compare  their  own  contributions  to  those  of  their  peers,  and 
the  affirmation  of  their  relative  standing  in  the  class  may  be  powerful  motivation  for  learning  (Hurlburt,  2008). 

Thus,  Web  2.0  technologies  has  the  ability  to  “support  active  and  social  learning,  provide  opportunities  and  venues 
for  student  publication,  provide  opportunities  to  provide  effective  and  efficient  feedback  to  learners,  and  provide 
opportunities  to  scaffold  learning  in  the  student’s  Zone  of  Proximal  Development”  (Hartshorne  & Ajjan,  2009; 
Vygotsky,  1978).  In  addition,  Web  2.0  provides  numerous  opportunities  for  social  interactions  and  collaboration 
among  students,  teachers,  subject  matter  experts,  professionals  in  different  fields,  as  well  as  a host  of  others  with 
related  interests. 

The  pedagogical  benefits  of  Web  2.0  have  been  well  documented  in  the  literature.  However,  most  of  the 
existing  studies  on  the  use  of  Web  2.0  technologies  in  teaching  and  learning  environments  have  been  anecdotal  in 
nature  or  in  the  form  of  case  studies.  Huang,  Yoo,  and  Choi’s  (2008)  study,  for  example,  found  that  correlation 
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exists  between  learning  style  and  learners’  preferences  and  attitudes  towards  using  Web  2.0  technologies.  Lambert 
and  Kidd’s  (2008)  explored  the  potential  impact  of  the  design  of  Web  2.0  environments  on  cognitive  load.  While 
such  studies  are  collectively  useful  in  providing  a broader  view  of  issues  surrounding  instructional  uses  of  Web  2.0 
technologies,  they  are  limited  in  scope,  as  they  address  such  issues  within  the  context  of  one  or  two  courses. 

Today’s  students  are  “digital  natives”  (Prensky,  2007),  and  make  increasing  use  of  Web  2.0  technologies 
in  their  daily  lives.  The  vast  majority  of  educators,  on  the  other  hand,  still  have  little  or  no  experience  with  these 
new  tools.  Teachers  and  instructors  need  to  understand  what  opportunities  Web  2.0  tools  provide  for  teaching  and 
learning,  what  kinds  of  barriers  they  may  encounter  when  using  them,  and  how  to  effectively  implement  the  new 
tools  in  their  teaching. 

Among  barriers  to  effective  use  of  web  2.0  tools  for  instruction  is  the  challenge  of  creating  meaningful 
assignments  to  promote  desired  learning  outcomes.  Ill-designed  assignment  with  no  visible  connection  to  the  overall 
purpose  of  the  course  not  only  frustrates  students,  it  decreases  students’  interest  in  using  the  tool,  and  results  in  little 
or  no  learning  (Reynard,  2009).  Moreover,  educators  needs  to  be  conversant  of  the  fact  that  the  social  nature  of  Web 
2.0  tools  which  makes  them  attractive  as  potential  learning  tools  is  also  one  of  its  drawbacks.  “The  very  key  to  the 
success  of  a blog  assignment  structure,  then,  is  fundamentally  counterintuitive.  In  order  to  take  advantage  of  the 
virtual  social  space  that  is  spontaneously  created  in  the  natural  blogosphere,  course  work  must  dictate  the  precise 
level  of  engagement  of  the  participants,  must  make  the  class  blogosphere  entirely  unnatural  . . . not  spontaneously 
social...”  (Hurlburt,  2008,  p.  184).  According  to  Hurlburt  (2008),  there  are  “invisible  aspects”  to  facilitating 
learning  in  the  social  network  environments.  In  addition  to  the  visible  aspect  of  designing  effective  learning 
environments  with  these  tools,  educators  also  need  to  serve  as  coach,  mentor,  cheerleader,  as  well  as  task  master  to 
their  students  when  the  expected  learner  participation  does  not  materialize.  The  affective  aspects  of  teaching  with 
Web  2.0  tools  may  provide  both  opportunities  and  as  well  as  barriers  that  are  yet  to  be  explored. 

Currently  little  research  has  explored  these  issues.  Therefore,  this  study  attempts  to  provide  a first  step  in 
understanding  the  benefits  of  and  barriers  to  using  Web  2.0  technologies,  especially  in  higher  education,  and  in 
developing  guidelines  for  teaching  with  Web  2.0  technologies. 

Research  Questions 

The  purpose  of  this  study  was  to  explore  best  practices  in  teaching  with  Web  2.0  technologies  as  well  as 
the  benefits  and  barriers  associated  with  the  use  of  Web  2.0.  Research  questions  guiding  this  study  are  as  follows: 

(1)  What  are  the  benefits  of  using  Web  2.0  technologies  in  teaching? 

(2)  What  are  the  barriers  to  using  Web  2.0  technologies  in  teaching? 

(3)  What  are  the  best  practices  in  teaching  with  Web  2.0  technologies? 

Research  Method 

A Web-based  survey  was  undertaken  in  November  2008.  The  first  author  sent  out  an  email  invitation, 
including  a link  to  the  web-based  survey,  to  a number  of  university  instructors  who  were  known  to  have 
considerable  experience  in  teaching  with  Web  2.0  technologies.  The  survey  included  four  demographic  questions 
and  10  open-ended  questions.  The  key  questions  include  the  following: 

• Describe  1-2  ways  you  use  Web  2.0  technologies  in  your  teaching. 

• What  are  the  benefits  of  using  Web  2.0  technologies  in  teaching? 

• What  are  the  barriers  to  using  Web  2.0  technologies  in  your  teaching? 

• What  instructional  strategies  or  techniques  have  already  worked  for  you?  Please  describe  best  practices  in 
teaching  with  Web  2.0  technologies  based  on  your  direct  or  indirect  experiences. 

• What  has  NOT  worked  for  you  in  terms  of  Web  2.0  technologies  in  teaching  and  learning? 

• What  would  be  effective  ways  to  use  Web  2.0  technologies  in  teaching  that  you  have  yet  to  try  but  are 
perhaps  thinking  about? 

Fourteen  university  instructors  participated  in  this  study.  The  participants,  consisting  of  64.29%  female  and 
35.71%  male,  were  from  a number  of  different  universities  in  the  United  States.  They  ranged  in  age  from  30’s  to 
60’s  (31-35:  7.14%;  36-40:  21.43%;  41-45:  14.29%;  46-50:  21.43%;  51-55:  7.14%;  56-60:  21.43%;  61-65:  7.14%). 
The  instructors  had  an  average  of  15  years  of  teaching  experience  and  an  average  of  3.71  years  ofusing  Web  2.0 
technologies  in  teaching. 

Qualitative  data  from  the  Web-based  survey  were  analyzed  using  the  constant  comparative  method  (Glaser 
& Strauss,  1967;  Strauss  & Corbin,  1990).  All  responses  were  carefully  read  and  reread  and  were  coded  and 
constantly  compared  to  other  data.  In  the  process,  some  coded  data  were  renamed  or  merged  into  new  categories.  A 
number  of  categories  or  themes  were  identified  and  were  organized  under  three  overarching  categories  that  this 
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study  focused  on:  (1)  benefits  of  using  Web  2.0  technologies  in  teaching,  (2)  barriers  to  using  Web  2.0  technologies 
in  teaching,  and  (3)  best  practices  and  tips  for  teaching  with  Web  2.0  technologies. 

Results 

The  survey  results  provided  insightful  guidelines  and  tips  for  using  Web  2.0  technologies  in  teaching.  Web 
2.0  technologies  used  by  the  participants  include:  blogs,  Wikis,  Youtube,  social  bookmarking,  podcasts,  webcasts, 
Facebook,  Myspace,  Flickr,  Twitter,  Skype,  Second  Life,  and  Tegrity.  The  participants  reported  how  they  used  these 
Web  2.0  technologies  in  their  teaching,  what  worked  well,  what  did  not  work  well,  and  so  forth.  The  discussion  of 
results  is  organized  around  the  following  three  focus  areas:  (1)  benefits  of  using  Web  2.0  technologies  in  teaching, 
(2)  barriers  to  using  Web  2.0  technologies  in  teaching,  and  (3)  best  practices  and  tips  for  teaching  with  Web  2.0 
technologies. 

Benefits  of  using  web  2.0  technologies  in  teaching 

The  study  results  indicate  that  the  major  benefits  of  using  Web  2.0  technologies  in  teaching  include  (1) 
interaction,  communication  and  collaboration,  (2)  knowledge  creation,  (3)  ease  of  use  and  flexibility,  and  (4)  writing 
and  technology  skills. 

Interaction,  communication  and  collaboration.  Most  participants  believed  that  using  Web  2.0 
technologies  in  teaching  helps  build  a sense  of  community,  increases  interaction  and  communication  among  the 
instructor,  students,  and  other  people,  and  promotes  collaboration  and  resource  sharing.  The  following  are  some  of 
the  comments  by  the  participants: 

• “I  think,  if  used  correctly,  they  can  help  develop  a better  sense  of  connectivity  between  students  and 
teachers  and  afford  students  opportunities  to  connect  and  communicate  with  classmates  and 
resources  throughout  the  world. . . ” 

• “They  reduce  the  distance  between  teacher  and  students.  ” 

• “Students  learn  about  new  ways  of  collaboration.  ” 

• “Students  and  teachers  see  learning  as  a more  social  process.  It's  not  just  the  book  and  yourself;  it's 
collaborative  meaning  making.  ” 

Knowledge  creation.  50%  of  the  participants  reported  that  Web  2.0  technologies  enable  students  to 
“become  creators  of  knowledge.”  As  one  noted,  Web  2.0  technologies  give  students  “the  opportunity  to  create 
content  themselves  instead  of  just  listening  to  lectures,”  and  this  supports  active  and  student-centered  learning  in 
which  students  take  responsibility  for  their  learning.  Several  participants  also  noted  that  Web  2.0  technologies  create 
an  environment  where  a teacher  becomes  a facilitator  of  learning  rather  than  a distributor  of  knowledge. 

Ease  of  me  and  flexibility.  About  a third  of  the  participants  reported  that  Web  2.0  tools  are  easy-to-use 
and  flexible.  They  noted  that  while  some  of  the  traditional  course  management  systems  (CMS)  are  too  static,  Web 
2.0  tools  remove  time  constraints  by  providing  a more  flexible  learning  environment  that  is  not  inhibited  to 
classroom  walls. 

'Writing  and  technology  skills.  Several  participants  noted  that  the  use  of  Web  2.0  technologies  help 
students  become  more  proficient  in  writing  and  in  the  application  of  technology. 

In  addition  to  these  four  major  benefits,  the  participants  also  mentioned  that  using  Web  2.0  technologies 
“helps  teachers  understand  a little  more  about  the  world  of  their  students,”  and  “motivates  the  students.” 

Barriers  to  using  web  2.0  technologies  in  teaching 

The  study  results  indicate  that  the  major  barriers  university  instructors  encounter  in  teaching  with  Web  2.0 
technologies  are  (1)  uneasiness  with  openness,  (2)  technical  problems,  and  (3)  time. 

Uneasiness  with  openness.  A number  of  participants  noted  that  the  open  nature  of  Web  2.0  technologies  is 
still  new  to  many  students.  They  reported  that  some  students  are  very  uncomfortable  with  the  openness  and  are 
reluctant  to  participate  in  class  activities  that  utilize  Web  2.0. 

. . . And  the  immediate  and  public  nature  of  wiki  collaboration  made  some  of  my  students  feel  more  self- 

conscious  and  a bit  uneasy  at  times...  These  students  preferred  one-to-one  teacher-student  interaction  more 

than  public,  peer-to-peer  interactions. . . 

Technical  problems.  Five  participants  reported  that  students  who  have  older  computers  often  have 
technical  issues  when  using  Web  2.0  tools.  It  was  also  noted  that  some  Web  2.0  tools  are  “still  a little  primitive,” 
having  technical  glitches  and  might  not  work  well  with  current  course  management  systems.  Several  participants 
mentioned  that  universities  do  not  provide  enough  technical  support  for  faculty  who  are  unfamiliar  with  Web  2.0 
technologies. 
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Time.  It  takes  time  to  learn  and  manage  new  technologies.  Time  was  another  barrier  identified  in  this  study. 
Several  participants  reported  that  learning  new  technologies  takes  time  away  from  learning  subject  matter  content. 

Best  practices  and  tips  for  teaching  with  web  2.0  technologies 

As  mentioned  earlier,  the  participants  in  this  study  were  asked  how  they  have  used  Web  2.0  technologies 
in  their  teaching,  what  instructional  strategies  or  techniques  have  worked  well,  and  what  has  not  worked  for  them. 
Based  on  data  analyses,  the  following  best  practices  or  guidelines  were  identified. 

Do  NOT  introduce  too  many  technologies  new  to  students  in  one  semester.  A number  of  participants 
indicated  that  using  too  many  Web  2.0  technologies  in  one  semester  could  lead  to  surface  learning.  They  suggested 
that  instructors  utilize  a small  number  of  the  tools  adding  more  only  as  expertise  is  developed. 

Do  NOT  use  multiple  technologies  that  do  the  same  thing.  Several  participants  suggested  that  instructors 
should  not  introduce  more  than  one  application  that  does  the  same  thing.  It  was  noted  that  students  must  often 
manage  several  email  accounts  and  forums  and  a new  technology,  if  used  like  an  existing  tool,  simply  creates 
management  problems. 

Students  and  I also  felt  at  times  that  Wiki  space  is  just  another  forum  to  manage  in  addition  to  several 
email  accounts,  WebCT,  and  etc  in  this  busy  life.  The  fact  that  most  of  my  students  were  working  full  time 
and  had  several  email  accounts  and  other  "online  message  boards"  to  check  added  to  that  feeling,  perhaps. 
Provide  appropriate  instruction,  tutorials,  examples,  and  frequent  feedback.  A number  of  participants 
emphasized  the  importance  of  providing  in-class  instruction,  tutorials,  and  examples  that  teach  how  to  use  Web  2.0 
technologies.  They  pointed  out  that  Web  2.0  technologies  are  still  new  to  many  students,  and  support  is  needed  if 
using  them  for  learning  activities  and  projects.  On  the  other  hand,  it  was  also  noted  that  instructors  should  help  their 
students  to  become  independent  learners  in  keeping  abreast  of  emerging  technologies. 

“We  can  teach  them  how  to  use  the  tools  available  today,  but  by  the  time  they  graduate  from  the  U.,  there 
will  be  new  tools  and  no  one  to  teach  them.  So,  right  up  front  they  need  to  learn  how  to  find  resources  on 
the  web  to  help  them  learn  new  technology.  ” 

Facilitate  collaborative  learning.  The  participants  in  this  study  reported  a variety  of  strategies  for 
facilitating  collaborative  learning  using  Web  2.0  technologies.  Examples  of  the  strategies  include  the  following: 

• Using  wikis  for  collaborative  writing  projects. 

• Using  a blog  as  a collaborative  reflection  space  beyond  personal  journals  by  requiring  students  to 
respond  and  provide  feedback  to  each  other. 

• Using  a social  bookmarking  site  for  sharing  resources. 

• Using  peer  evaluation. 

Build  a sense  of  community  in  your  classroom  first  before  trying  more  public  collaboration.  Web  2.0  is 
generally  characterized  by  openness,  social  interaction  and  collaboration.  Not  surprisingly,  some  of  the  participants 
used  Web  2.0  tools  for  inter-institutional  collaboration.  One  reported  that  her  students  wrote  a wikibook  on  learning 
theories  in  collaboration  with  students  in  another  university.  She  believed  that  their  inter-institutional  collaboration 
was  not  very  successful  because  the  students’  entry  level  knowledge  and  focus  (research  vs.  practice)  were  different. 
She  also  mentioned  that  some  students  were  overwhelmed  by  trying  new  technologies  with  new  people.  She 
contended  that  instructors  “have  to  establish  at  least  a certain  sense  of  community  first  before  trying  more  public 
collaboration  and  social  interaction.” 

Other  strategies  for  teaching  with  Web  2.0  technologies  the  participants  suggested  include:  creating  an 
engaging  and  supportive  environment,  providing  clear  goals  and  objectives  for  using  Web  2.0  technologies, 
rewarding  students  for  good  work  and  good  contributions,  and  showing  YouTube  videos  to  start  or  end  class. 

Discussions 

Web  2.0  has  been  one  of  the  major  topics  in  professional  conferences  and  journals.  Teachers  and 
instructors  in  diverse  contexts  are  exploring  ways  to  use  Web  2.0  technologies  in  teaching  and  learning.  Many  case 
studies  on  integrating  Web  2.0  technologies  in  teaching  and  learning  are  providing  valuable  tips  and  lessons. 
However,  Web  2.0  is  still  new  to  a majority  of  instructors,  especially  in  higher  education  settings.  This  study 
attempted  to  present  a first  step  in  understanding  the  benefits  of  and  barriers  to  using  Web  2.0  technologies  and  in 
developing  guidelines  for  teaching  with  Web  2.0  technologies. 

Web  2.0  has  the  potential  to  provide  more  interactive  and  customized  learning  environments  where 
students  create  knowledge,  rather  than  passively  receive  information  from  instructors,  interact  and  collaborate  with 
those  who  have  similar  interests  globally,  and  obtain  opportunities  to  learn  to  become  professionals  in  communities 
of  practice.  However,  it  appears  that  many  teachers  and  instructors  are  not  using  Web  2.0  technologies  to  their 
potentials.  Instead  of  maximizing  the  benefits  of  Web  2.0,  educators  often  do  the  same  thing  with  a new  tool  much 
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like  early  distance  education  instructors  who  simply  moved  their  course  content  to  the  Web  without  adapting  the 
course  and  teaching  methods  to  the  new  environment. 

Instructors  should  keep  in  mind  that  Web  2.0  itself  does  not  guarantee  more  effective  learning  and 
teaching.  Simply  adding  Web  2.0  tools  to  our  traditional  teaching  practice  cannot  realize  the  potential  benefits  Web 
2.0.  New  technologies  can  help  us  improve  our  teaching  and  learning  only  when  they  are  used  effectively  with  clear 
goals  and  proper  methods  (Reynard,  2009;  Hurlburt,  2008).  The  effective  use  of  new  technologies  requires 
innovation  in  teaching  methods. 

Participants  in  this  study  reported  that  universities  do  not  provide  enough  technical  support  for  faculty  who  are 
unfamiliar  with  Web  2.0.  We  agree  that  it  is  important  to  provide  faculty  with  appropriate  technical  support. 
However,  we  believe  that  it  is  more  critical  to  help  instructors  develop  new  ways  of  teaching  that  reflect  the 
capabilities  of  the  new  tools  and  their  potentials  rather  than  simply  teach  them  how  to  use  the  tools.  In  addition, 
institutional  reward  structure  that  is  commensurate  with  the  time  and  efforts  that  instructors  invest  in  exploring  these 
tools  will  likely  remove  some  of  the  barriers  to  instructional  uses  of  Web  2.0  tools. 

Students  also  require  some  support  to  effectively  learn  with  Web  2.0  technologies.  Today’s  students 
integrate  technology  in  their  everyday  lives  and  they  are  constantly  connected  to  their  friends,  family,  and  various 
resources  via  technology.  Therefore,  it  is  easy  to  assume  that  they  are  technologically  savvy.  However,  as  Oblinger 
(2008)  contends,  “Not  all  students  have  computers,  not  all  are  skilled  users,  and  not  all  want  to  use  technology”  (p. 
18).  This  is  consistent  with  the  findings  in  this  study.  The  participants  noted  that  many  students  have  technical  issues 
with  old  computers,  are  not  very  comfortable  with  Web  2.0  technologies,  and  need  instructions  and  examples  on 
how  to  use  them.  In  teaching  with  Web  2.0  technologies,  instructors  should  be  prepared  to  provide  appropriate 
support  and  scaffolding.  Providing  step-by  step  procedural  guidance  might  be  necessary  in  some  situations,  but  we 
believe  that  instructors  should  also  help  their  students  find  appropriate  resources  and  teach  themselves  how  to  use 
technologies  since  instructors  cannot  provide  the  tutorials  forever. 

Openness  and  privacy  is  another  issue  instructors  should  consider  in  using  Web  2.0  technologies  in  their 
teaching.  As  noted  in  this  study,  some  students  are  not  comfortable  with  the  open  and  public  nature  of  Web  2.0  tools 
often  preventing  them  from  participating  in  learning  activities.  Openness  is  one  of  the  key  characteristics  of  Web  2.0 
and  is  expected  to  be  increasingly  reinforced  in  the  future.  There  should  be  more  research  on  this  issue  to  help 
instructors  effectively  prevent  potential  problems  and  counter  students’  uneasiness  with  openness. 

Limitations  and  Future  Research 

The  benefits,  barriers,  and  best  practices  identified  in  this  study  are  based  on  Web-based  survey  responses 
from  only  fourteen  university  instructors  even  though  they  had  considerable  experience  in  teaching  with  Web  2.0 
technologies  and  provided  great  insights  into  the  use  of  the  new  tools  in  higher  education  settings.  Further  studies 
might  survey  a large  number  of  instructors  and  also  conduct  interviews  in  addition  to  the  survey  with  hopes  of  better 
understanding  participants’  experiences.  Second,  it  might  be  interesting  to  observe  a number  of  courses  that  use 
Web  2.0  technologies  and  learn  about  students’  perspectives  as  well  as  those  of  instructors.  Finally,  future  research 
should  also  explore  how  Web  2.0  technologies  are  used  in  different  contexts,  including  K-12,  higher  education, 
corporate  settings,  and  the  comparison  and  contrast  of  the  findings. 
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Abstract 

To  best  design  technology  integration  professional  development,  one  must  have  a clear  understanding  of  teachers’ 
attitudes  and  beliefs  regarding  its  practice.  Students  enrolled  in  Learning  with  Technology,  a required  course  in  an 
online  Masters  in  Education  degree  program,  completed  a post-course  reflection  assignment.  Students  were 
practicing  teachers  or  certified  education  professionals.  An  analysis  of  225  responses  to  an  essay  question  regarding 
attitude  towards  technology  integration  is  shared.  Results  indicated  the  majority,  52%,  had  positive  feelings  about 
and  were  integrating  technology,  28%  had  positive  feelings  but  cited  obstacles  to  integration,  13%  were  fully 
integrating,  and  7%,  were  not  integrating.  The  most  common  obstacles  to  integration  included  skills,  efficacy, 
access,  and  time.  Implications  for  the  design  of  technology  integration  professional  development  activities  are 
shared. 

Background 

“Despite  a steady  wave  of  how-to  workshops  and  some  longer-duration  seminars,  infusing  technology  into 
curriculum  and  teaching  practices  remains  elusive  for  many  teachers.”  (Plair,  2008,  p. 70).  Compared  to  the 
availability  of  technology,  technology  integration,  defined  by  Lambert,  Gong,  and  Cuper  (2008)  as  “teachers 
utilizing  content  and  technological  and  pedagogical  expertise  effectively  for  the  benefit  of  students’  learning”  (p. 
386),  is  often  deficient  in  today’s  classrooms  (National  Center  for  Education  Statistics,  2000).  According  to  the 
National  Center  for  Education  Statistics  (2000),  99  percent  of  full-time  public  school  teachers  have  computer  and 
Internet  access  in  their  schools,  but  only  39  percent  integrate  technology  into  their  lessons.  In  general,  teachers  are 
comfortable  using  technology  for  personal  use  or  developing  instructional  materials,  but  they  do  not  integrate  it  into 
instruction  (Ertmer,  2005).  If  access  is  not  the  problem,  what  leads  to  inadequate  integration  and  what  can  be  done 
about  it? 

Various  factors  have  been  found  to  explain  teachers’  integration,  or  lack  thereof;  most  relate  to  existing  beliefs  and 
attitudes.  Hall  and  Martin  (2008),  for  example,  ascribe  the  lack  of  integration  to  self-efficacy.  Their  research 
revealed  a statistically  significant  correlation  between  teachers’  self-efficacy  beliefs  regarding  computer  skills  and 
their  integration  of  technology  into  instruction.  Similar  findings  were  uncovered  by  Piper  (2003).  Others  attribute 
teachers’  integration  to  existing  beliefs  about  teaching  and  learning  (Frederick,  Schweizer,  and  Lowe,  2006;  Kanaya, 
Light,  and  McMillan  Culp,  2005;  Tondeur,  Hermans,  van  Braak,  and  Valcke,  2008).  Kanaya  et  al.  (2005),  for 
example  found  a direct  relationship  between  perceived  benefits  of  technology  integration  and  existing  pedagogical 
beliefs.  Chen  (2008),  on  the  other  hand,  assigned  the  lack  of  integration  to  perceived  external  factors  and  to  limited 
or  incorrect  understanding  of  constructivist  instruction.  He  states,  “Teachers  may  have  incomplete  or  incorrect 
understanding  of  proposed  ideas  and  may  hold  conflicting  beliefs  without  noticing  the  inconsistency”  (2008).  For 
example.  Loveless  (2003)  found  that  teachers  were  in  support  of  technology,  but  believed  that  technology  should  be 
taught  as  its  own  subject.  How  does  one  modify  these  conflicting  beliefs  and  correct  or  complete  these 
understandings? 

One  way  to  improve  the  rate  of  integration  is  via  professional  development  (Brinkerhoff,  2006).  Professional 
development  provides  teachers  with  opportunities  to  explore  new  technologies  and  develop  new  skills  (Oncu, 
Delialioglu  and  Brown,  2008).  The  support  and  guidance  offered  in  professional  development  helps  teachers  to 
develop  the  confidence  to  practice  those  skills  (Brinkerhoff,  2006;  Overlaugh  and  Lu,  2008).  Also  to  be  considered 
are  the  words  of  Ertmer  (2005)  who  states,  “If  beliefs  are  formed  through  personal  experience,  then  changes  in 
beliefs  might  also  be  facilitated  through  experience.”  By  providing  professional  development  for  teachers  to  learn 
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about  and  practice  technology  integration,  teachers  may  come  to  correct  conflicting  or  complete  incomplete 
understandings. 

Even  with  the  best  of  intentions,  however,  professional  development  is  unlikely  to  be  successful  without  a pre- 
assessment of  teachers’  attitudes  and  beliefs.  “A  greater  focus  on  assessing  teachers’  attitudes  and  beliefs,” 
according  to  Hall  and  Martin  (2008),  “during  professional  development  could  result  in  a higher  transfer  of  skills  and 
knowledge  to  classroom  practice”  (p.  3).  Content  and  practice  alone  cannot  make  up  for  the  delivery  of  a training 
mismatched  to  its  audience.  Professional  development  programs  should  be  designed  to  complement  and  build  upon 
existing  attitudes  and  beliefs. 

Research  Questions 

Having  a clear  grasp  of  teachers’  preexisting  attitudes  and  beliefs  could  help  trainers  to  design  more  effective 
professional  development  activities  (King,  2002;  Windschitl  and  Sahl,  2002).  The  current  study  focused  on  the 
identification  of  teachers’  attitudes  and  beliefs  towards  technology  integration.  In  addition,  the  author  sought  to 
uncover  perceived  obstacles.  The  purpose  of  the  present  study  was  to  address  the  following  research  questions. 

1.  What  are  teacher’s  beliefs  and  attitudes  towards  technology  integration? 

2.  What  obstacles  prevent  teachers  from  integrating  technology  into  instruction? 

While  question  #2  was  not  directly  asked,  the  author  anticipated  that  this  issue  would  surface  in  response  to  question 

#1. 


Data  Collection 

The  analysis  was  conducted  with  data  collected  from  May  2008,  end-of-course  reflections  submitted  in  Learning 
with  Technology,  a required  course  in  an  all  online  Masters  in  Education  degree  program  at  a Midwestern  university. 
The  assignment  was  part  of  the  regularly  assigned  curriculum  and  required  students  to  respond  in  100  words  or  less 
to  the  question  “What  was  your  attitude  towards  technology  before  you  took  this  course?” 

Prior  to  the  course,  the  author  knew  she  was  going  to  analyze  the  reflections,  so  she  inserted  into  the  directions  a 
declaration  of  intended  research  and  a waiver.  The  declaration  stated: 

The  College  of  Education  is  conducting  a qualitative  research  study  about  teachers'  use  of  technology  in 
their  classrooms.  All  responses  will  remain  anonymous,  and  associations  will  not  be  made  to  your  place  of 
work.  You  contribution  will  be  used  to  help  teacher  education  programs  better  understand  technology  use 
strengths,  weakness,  achievements,  and  areas  of  need. 

The  waiver  stated:  “I  authorize  the  College  of  Education  to  use  my  responses  on  this  assignment  as  part  of  its 
qualitative  research  study.” 

At  the  time  this  course  was  offered,  the  college  offered  two  degree  programs,  both  leading  to  a master’s  in 
education.  Programs  were  designed  such  that  all  cohorts  took  the  same  courses  at  the  same  time.  In  the  spring  term 
of  2008,  there  were  1346  students  enrolled  in  the  course,  thus  offering  a large  sample  from  which  to  draw.  Of  those 
students  enrolled,  500  turned  in  this  last  assignment  (worth  5%  of  their  grade).  Because  the  degree  programs  are 
completely  online,  all  essays  were  submitted  electronically  for  grading.  A review  of  the  submissions  indicated  450 
initialled  the  waiver,  149  did  not,  and  5 submitted  blank  assignments.  A random  sampling  equal  to  50%  (225)  of 
submitted,  waiver  initialled  assignments  were  analyzed. 

Analysis  and  Summary 

To  group  responses,  the  author  used  a web-based  text  visualization  tool  called  Wordle  (www.wordle.net).  To  use 
Wordle,  one  simply  drops  text  into  the  online  text  box  and  a word  collage  image  appears.  The  image  allows  users  to 
see  how  frequently  words  appear  in  a given  text,  drawing  each  word  at  a size  proportional  to  its  frequency.  In  other 
words,  the  size  of  a word  is  proportional  to  that  word’s  frequency.  All  sampled  essay  responses  were  dropped  into 
the  Wordle  text  box.  The  most  frequently  cited  words  were  noted  and  used  to  develop  coded  categories: 

• 1A  Didn’t  care,  didn’t  do 

• 2A  Cared,  but  saw  obstacles  so  didn’t  do 

• 3 A Cared,  but  just  used  myself 

• 4A  Cared,  was  doing  some  integration(learning  from) 

• 5A  Cared,  was  doing  integration  (learning  with) 
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Regarding  categories  4A  and  5A,  many  respondents  cited  a report  from  their  required  readings  called  Laptops  for 
Learning:  Final  Report  and  Recommendations  of  the  Laptops  for  Learning  Task  Force  (Barrios  et  al.,  2004).  This 
report  distinguishes  between  “I  earning /row”  technology  and  “learning  with ” technology.  Learning  from  technology 
“is  akin  to  the  old  ‘sage  on  a stage’  notion  of  teaching.  The  technology  is  used  solely  to  deliver  or  broadcast 
information  to  students”  (p.  6).  Learning  with  technology,  on  the  other  hand,  “empowers  students  with  the  tools  to 
take  responsibility  for  their  own  learning”  (p.6)  and  it  “requires  a higher  level  of  thinking  and  problem  solving”  (p. 

7).  Because  this  distinction  was  often  explicated,  it  was  built  into  the  coding. 

As  a result  of  this  analysis,  another  theme  emerged:  obstacles  to  integration.  Most  respondents  indicated  a number  of 
obstacles  that  lead  to  their  attitude  and/or  prevented  them  from  integrating  more  technology.  The  author  grouped 
and  coded  these  responses  as  well.  These  categories  included: 

• IB  Lack  of  knowledge  about/skills  to  provide 

• 2B  Lack  confidence/efficacy  to  use 

• 3B  Lack  of  resources/access 

• 4B  Not  enough  time  to  learn 

• 5B  Students  are  too  young  or  unskilled 

• 6B  Non-supportive  administration 

Results 

Audience 

Participants  were  practicing  teachers  or  certified  education  professionals,  including  librarians  and  social  workers. 

The  majority,  191  (85%),  were  employed  by  urban  school  districts;  the  remaining  were  employed  by  suburban 
school  districts.  While  there  was  a distributed  sampling  of  grade  levels  at  which  these  professional  taught/ worked, 
the  majority,  96  (42.6%),  were  at  the  elementary  school  level.  Of  the  remaining,  15  (6.6%)  were  at  the  pre- 
kindergarten/ kindergarten  level,  39  (17.3%)  were  at  the  middle  school  level,  34  (15.1%)  were  at  the  high  school 
level,  and  41  (18.2%)  did  not  indicate. 

In  regards  to  specialty  areas,  a wide  variety  of  subject  areas  taught  was  represented:  librarian  (1),  math/science  (1), 
nurse  (1),  technology  (1),  vocational  (1),  language  arts/science  (2),  foreign  language  (3),  language  arts/social  studies 
(4),  art  (5),  music  (5),  physical  education  and/or  health  (5);  English,  reading,  or  language  arts  (6),  social  studies  (8), 
science  (13),  math  (15),  special  education  (28),  and  generalists  teaching  most  subject  areas  (83).  The  42  students 
who  did  not  indicate  their  grade  level  also  did  not  indicate  their  specialty  area. 

Some  demographics  - age,  gender,  race  -could  not  be  determined  because  they  were  not  requested.  The 
demographics  for  all  students  enrolled  in  the  course  at  that  time,  however,  could  be  determined  based  on  college 
enrolment  data.  The  gender  report  indicated  that  95%  (1285)  were  female,  4%  (47)  were  male,  and  1%  (14)  did  not 
report  their  gender.  The  ethnicity  report  indicated  that  70%  (944)  were  White,  non-Hispanic,  13%  (177)  were 
African-American,  6%  (82)  were  Hispanic,  7%  (100)  did  not  indicate  their  race/ethnicity,  2%  (29)  Asian  or  Pacific 
Islander,  <1%  (6)  were  “other”,  <1%  (3)  was  multi-ethnicity,  and  <1%  (1)  was  American  Indian  or  Alaskan  Native. 
The  age  report  indicated  that  8%  (106)  of  students  were  20-  30  years  of  age,  13%(181)  were  31-40  years  of  age, 

10%  (128)  were  41-50  years  of  age,  11%  (152)  were  51-60  years  of  age,  18%  (24)  were  61-70  years  of  age,  and 
<1%  (1)  was  71  plus  years  of  age. 

Attitude  towards  Technology  Prior  to  Instruction 

A review  of  responses  to  the  question,  “What  was  your  attitude  towards  technology  before  you  took  this  course?” 
revealed  a varied,  but  not  equal  distribution  of  attitudes  towards  technology  integration  (See  Figure  1).  The 
majority,  52%  (117),  was  using  some  technology  in  the  classroom,  but  the  level  of  integration  would  be  best 
described  as  “learning/row”  technology,  rather  than  “learning  with”  technology.  The  typicality  of  a “learning 
from”  behavior  mirrors  findings  in  a 2003  U.S.  Department  of  Education’s  Integrated  Studies  of  Educational 
Technology  report.  That  report  indicated  the  computer  related  activities  most  often  exercised  by  teachers  with  their 
students  included  writing,  improving  computer  skills,  doing  research  using  the  Internet,  using  computers  as  a free- 
time or  reward  activity  and  doing  practice  drills. 
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Figure  1 - Attitude  towards  Technology  Integration  Prior  to  Instruction 


Examples  of  responses  at  a “learning  from”  level  are: 

• “Before  I started  this  course,  I must  admit  that  I inadvertently  focused  more  on  learning  from  technology  than 
learning  with  technology.  I would  assign  Internet  projects  to  my  students  that  wouldn't  really  teach  them 
much.” 

• “This  class  opened  up  my  eyes  in  regards  to  what  I was  viewing  as  ‘embracing  technology’  in  my  classroom. 
What  1 thought  was  sufficient,  wasn't  really  sufficient  at  all.” 

• “Prior  to  taking  this  course,  I felt  pretty  confident  of  my  technology  skills.  Little  did  I know  that  I wasn't  even 
hitting  the  rim  of  the  basket” 

• “I  felt  technology  was  an  important  aspect  of  my  students'  lives.  Even  with  these  thoughts,  I found  it  difficult  to 
integrate  the  technology  into  my  lessons  without  changing  everything  completely.” 

• “I  don’t  think  I was  ever  resistant  to  technology,  just  unaware  of  all  it  can  do  for  me  and  a teacher  and  my 
students  in  their  learning.  “ 

The  comments  indicated  that  while  these  students  (teachers)  thought  that  they  were  adequately  integrating 
technology,  they  came  to  realize  they  could  have  been  doing  much  more.  It  is  also  apparent  some  of  their 
misconceptions  may  have  stemmed  from  a lack  of  knowledge  about  how  to  go  about  integrating  technology  at  a 
“learning  with ” level. 

The  next  most  largely  represented  attitude  was  that  technology  integration  was  important,  but  due  to  perceived 
obstacles,  they  did  not  put  that  belief  into  practice,  28%  (63).  Obstacles  cited  ranged  from  non-supportive 
administration  to  lack  of  knowledge  and  skills.  These  obstacles  are  described  in  the  next  section. 

A small  percentage  of  respondents,  13%  (30),  were  fully  integrating  technology  at  a level  at  which  could  be 
described  as  “learning  with”  technology.  Examples  of  responses  are: 
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• “I  am  not  a digital  native  but  was  a migrant  to  it  long  before  this  course.  It  started  off  more  than  a decade  ago 
with  me.  I can  effectively  develop  a student  centered  inquiry  based,  self-directed  learning  environment  with 
equitable  assessment.” 

• “I  am  comfortable  using  technology  in  my  classroom  and  actively  plan  lessons  using  the  computer  as  a 
component  all  day  long.  I find  access  to  a computer  a wonderful  and  effective  way  to  reach  my  special 
education]  students.” 

• “The  essential  question  when  I begin  this  course  was  how  do  I guide  students  in  ways  that  put  them  in  charge  of 
showing  their  understanding  and  knowledge  through  activities  that  provide  the  context  for  authentic  learning, 
using  technology  and  at  the  same  time  addressing  the  learning  standards  required  by  the  district  and  the  state. “ 

In  reviewing  responses  such  as  the  one  of  above,  it  was  apparent  that  they  were  using  technology  to  empower  their 
own  students.  In  all  cases,  it  was  also  apparent  they  wished  they  could  be  doing  more. 

As  for  the  remaining  coded  attitudes,  “Cared,  but  just  used  myself’  and  “Didn’t  care,  didn’t  do”  were  expressed  by 
4%  (8)  and  3%  (7),  respectively,  of  the  remaining  sampling.  Examples  of  responses  are: 

• “I  felt  that  technology  was  a part  of  our  culture,  but  I guess  I did  not  see  the  real  relevance  of  technology  in  the 
classroom.” 

• “My  attitude  towards  technology  before  I took  this  course  was  negative.  I did  not  understand  why  it  was 
necessary  to  incorporate  technology  into  the  classroom.  I grew  up  with  limited  use  of  technology  tools  during 
my  elementary  and  secondary  education;  therefore,  I never  saw  it  as  a means  necessary  to  increase  student 
achievement  and  instructional  practices.” 

• “I  believe  my  attitude  towards  technology  prior  to  this  course  was  not  very  important.  I do  use  a computer  at 
home  and  at  work  to  create  documents  and  send  e-mails. . .” 

Respondents  appeared  to  use  technology  in  their  own  lives,  but  did  not  see  value  in  bringing  it  into  the  classroom. 
These  findings  were  consistent  with  Chen  (2008)  who  found  that  all  teacher  participants  used  various  technologies 
for  personal  use,  instruction  planning,  and  administrative  work,  but  very  few  used  technology  to  achieve 
instructional  goals  other  than  covering  curricular  content,  preparing  students  for  examinations,  and  highlighting 
important  concepts.  Fortunately,  in  the  current  study,  every  “didn’t  care,  didn’t  do”  or  “cared,  but  just  used  myself’ 
respondent  later  expressed  in  his/her  post-course  reflection  that  as  a result  of  what  was  learned  in  the  course,  he/she 
was  now  going  to  take  small  steps  to  integrating  technology.  This  change  reflects  a very  positive  outcome  of 
technology  integration  professional  development. 


Obstacles  to  Integrating  Technology 

Rogers  (1995)  states  people  are  more  likely  to  adopt  an  innovation  if  the  innovation  offers  a better  way  to  do 
something,  is  compatible  with  their  values,  beliefs,  and  needs,  is  not  too  complex,  can  be  tried  out  before  adoption, 
and  has  observable  benefits.  It  became  apparent,  when  reviewing  responses,  the  obstacles  were  rooted  in  factors 
conflicting  with  perceived  attributes.  The  most  commonly  cited  obstacles  were:  lack  of  knowledge  about/skills  to 
provide  (92),  lack  confidence/efficacy  to  use  (28),  lack  of  resources/access  (26);  not  enough  time  to  leam  (11), 
students  are  too  young  or  unskilled  (5),  and  non-supportive  administration  (3)  (See  Figure  2). 
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Figure  2 Obstacles  to  Integrating  Technology 


6B  Non-supportive administration 
5B  Students  are  too  young  or  unskilled 
4B  Not  enough  time  to  learn 
3B  Lackof  resources/access 
2B  Lack  confidence  to  use 
IB  Lackof  knowledge /skills 


Frequency  of  citation  (N=225) 


Some  respondents  reported  more  than  one  obstacle  and  others  reported  none.  Examples  of  responses  are  listed 
below.  Next  to  each  statement,  in  brackets,  is  the  obstacle(s)  coded: 

• “I  have  only  been  teaching  for  five  years,  and  have  always  thought  about  the  use  of  technology.  During  my  first 
two  years  I was  busy  learning  the  curriculum  and  trying  to  manage  twenty-eight  students.  I felt  like  I didn’t 
have  room  in  my  curriculum  for  technological  lessons.”  [Obstacles:  time;  knowledge/skills] 

• “I  was  negative  toward  technology  because  the  resources  that  are  available  to  me  at  my  school  are  limited.  The 
computers  in  my  classroom  are  old  and  run  very  slowly.  It  is  also  difficult  to  download  any  new  software  and  I 
only  have  Internet  access  on  one  computer.  I also  didn't  have  ideas  on  how  to  integrate  technology  in  my 
kindergarten  classroom.”  [Obstacles:  access;  young  students] 

• “Five  weeks  ago  if  someone  were  to  ask  me  about  technology  in  my  school,  I would  roll  my  eyes  and  reply, 
‘What  technology?  We  barely  have  computers.’  That  would  be  the  end  of  the  conversation  and  I would  move 
on  to  a new  topic.”  [Obstacles:  access] 

• “I  have  always  felt  that  1 have  the  basic  skills  needed  to  work  with  technology,  but  the  thought  of  having  my 
students  use  it  to  learn  scared  me!  I felt  like  I did  not  know  where  to  begin.”  [Obstacle:  confidence/efficacy]. 

• “I  was  unaware  that  standards  for  teaching  with  technology  even  existed.”  [Obstacle:  knowledge/  skills]. 

• “My  initial  attitude  toward  technology  before  I took  this  course  was  negative  because  the  word  technology 
reminded  me  of  my  deficiencies  as  an  educator.”  [Obstacle:  confidence/efficacy]. 

• I have  always  been  an  advocate  for  technology  and  new  experiences  [but],  I was  often  frustrated  by  the  schools 
limited  resources  as  well  as  lack  of  ideas.  [Obstacles:  access;  knowledge/skills] 

It  becomes  apparent  after  reviewing  these  comments  and  the  frequency  chart  that  reasons  vary  in  type  and  number. 
Many  are  consistent  with  perceived  obstacles  uncovered  by  Demetriadis,  Barbas  et  al.  (2003).  The  most  commonly 
obstacles  found  in  their  research  were:  (1)  material  conditions  (including  an  insufficient  number  of  computers  and 
insufficient  technology  expertise);  (2)  difficulty  integrating  technology  into  the  regular  curriculum  and  instruction; 
and  (3)  lack  of  supervisory  and  technical  staff.  Perceived  obstacles,  real  or  not,  should  be  addressed  in  the  design 
of  professional  development. 


Limitations 
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There  are  a variety  of  factors  that  could  keep  one  from  generalizing  the  results  of  this  analysis  to  other  populations. 
The  first  factor  that  could  influence  the  findings  is  that  students  (teachers)  enrolled  in  this  course  chose  to  earn  their 
degree  online.  Students  enrolling  in  an  online  degree  may  have  a more  favourable  attitude  towards  technology  than 
those  enrolling  in  an  on-campus  degree  program.  Another  factor  that  could  influence  the  findings  is  the  large 
percentage  of  students  (teachers)  who  work  for  an  urban  school  district.  Access  to  technology,  availability  of 
professional  development  resources,  and  the  study  body  demographics  in  urban  school  districts  can  vary  from  that 
found  in  suburban  or  rural  districts.  A third  factor  to  consider  is  the  timing  of  this  assignment,  in  this  course. 
Despite  the  assignment  directions  requesting  students  (teachers)  to  consider  their  attitudes  towards  technology 
integration  before  the  course,  their  perceptions  may  have  shifted  during  the  course.  A fourth  factor  to  consider  was 
the  large  percentage  of  females  enrolled  in  this  course  and  degree  problem.  It  is  possible  females  may  integrate 
technology  into  the  classroom  to  a greater  or  lesser  degree  than  males. 


Implications 

Teachers’  attitudes  and  beliefs  should  be  carefully  considered  in  the  design  of  technology  integration  professional 
development.  This  analysis  indicates  many  teachers  believed  technology  integration  was  important,  but  didn’t  know 
how  to  do  it.  It  was  also  uncovered  that  some  had  mixed  feelings,  some  of  which  were  rooted  in  perceived  obstacles 
related  to  skills,  self-efficacy,  access,  and  time.  Ideally,  the  majority  of  teachers  should  be  fully  integrating 
technology  in  their  classrooms.  Incorporating  skill-building  activities  and  suggestions  for  authentic  applications  into 
professional  development  would  help  to  build  teachers’  efficacy  towards  and  appreciation  for  technology 
integration. 

Technology  integration  professional  development  may  require  more  than  the  typical  lecture  and  drill  method  of 
instruction.  As  with  any  new  skill,  technology  integration  professional  development  should  delivered  in  a 
supportive  environment  emphasizing  authentic  practice.  Jan  Hawkins,  former  director  of  the  Center  for  Children  and 
Technology  at  the  Education  Development  Center,  Inc.  abides  by  four  primary  guidelines  when  combining 
professional  development  and  technology: 

• Intensive  sessions  where  teachers  are  able  to  explore  new  ideas  and  materials; 

• Follow-up  support  over  an  extended  period  of  time  with  mentors; 

• Ongoing,  reflective  conversations  with  colleagues  doing  the  same  job  and  trying  to  make  similar  changes; 
and 

• Observation  of  other  teachers  in  their  classrooms.  (1997,  p.  215) 

Despite  the  date  of  Hawkins’  article,  1997,  her  authentic,  collaborative  prescription  is  constructive  and  consistent 
with  Brinkerhoff  (2006)  and  Overlaugh  and  Lu  (2008).  She  emphasizes  a mode  of  delivery  that  develops  skills  and 
fosters  efficacy,  two  of  the  top  three  primary  obstacles  indicated  in  this  analysis’s  findings. 

Regarding  efficacy,  Linnenbrink  and  Pintrich  (2002)  indicate  its  development  is  facilitated  by  providing 
opportunities  for  success  within  one’s  range  of  capabilities  and  then  gradually  developing  new  skills.  This 
suggestion  is  in  line  with  Hawkins  (1997).  Self-efficacy  is  not  the  result  of  will-power,  but  actual  success  Beck 
(2004).  To  improve  self-efficacy  towards  a given  skill,  Weiner  (1986)  suggests  linking  behavioral  outcomes  such  as 
engagement  with  achievement.  This  means  professional  development  should  provide  opportunities  to  link  success 
to  strategy  use,  rather  than  just  effort. 

Regarding  access,  another  common  obstacle,  a teacher  may  want  to  integrate  technology,  but  due  to  a real  or 
perceived  lack  of  resource,  cannot.  For  example,  a respondent  stated: 

I have  always  held  a positive  attitude  toward  technology.  I use  the  Internet  regularly  in  my  personal  life, 
but  I did  little  to  integrate  it  into  my  classroom.  Part  of  the  reason  for  this  was  simply  a lack  of  access.  I 
did  not  have  a classroom  computer  until  two  months  ago. 
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Lack  of  access  or  resources  was  cited  by  twenty-six  respondents.  In  most  cases  lack  of  access  meant  having  only 
one  computer  in  their  classroom.  This  means  professional  development  should  include  tips  for  integrating 
technology  when  access  is  limited.  Kathy  Shrock  provides  a host  of  resources  on  her  Guide  for  Educators  website 
in  a section  entitled:  “The  One  Computer  Classroom:  A Review  of  the  Internet  Literature.”  Integrating  thematic 
station-based  work,  scheduling,  or  using  a projector,  are  three  simple  ways  to  overcome  the  one  computer,  one 
classroom  obstacle.  Strategies  like  these  could  be  incorporated  into  professional  development,  particularly  when 
access  is  a perceived  obstacle. 

Also  to  be  considered  when  addressing  access  is  students’  (of  the  teachers)  access  to  technology  at  home.  As  one 
respondent  stated  in  her  reflection,  “My  students  are  low-income  minorities  and  less  than  five,  when  asked,  have 
access  to  a computer  at  home.”  In  such  situations,  student  experiences  with  technology  are  more  likely  to  be 
concentrated  on  computer  basics  and  support  for  academic  remediation  then  on  using  technology  to  enhance  the 
learning  and  critical  thinking  process  (Warschauer,  Knobel,  and  Stone,  2004).  Teachers,  therefore,  must  be  trained 
how  to  teach  these  basic  skills  within  the  context  of  authentic,  rich  lessons. 

The  strength  of  obstacles  should  not  be  underestimated  because  nary  was  the  teacher  who  was  strongly  passionate 
about  technology  and  not  integrating  technology  at  a “learning  with”  level.  The  analysis  indicates  obstacles,  real  or 
not,  must  be  dealt  with  in  professional  development.  Heeding  existing  perceptions  and  attitudes  in  the  design  and 
delivery  of  professional  development  will  result  in  a higher  transfer  of  knowledge  and  skills  (King,  2002;  Martin 
and  Hall,  2007;  Windschitl  and  Sahl,  2002).  The  hope,  consequently,  is  that  improved  knowledge  and  skills  will 
lead  to  an  increased  percentage  of  teachers  who  help  their  students  to  learn  with  as  well  as  from  technology. 


Conclusions 

Ertmer  (2005)  indicates,  “If  we  truly  hope  to  increase  teachers’  uses  of  technology,  especially  uses  that  increase 
student  learning,  we  must  consider  how  teachers’  current  classroom  practices  are  rooted  in,  and  mediated  by, 
existing  pedagogical  beliefs”  (p.  36).  As  one  respondent  so  eloquently  put  it,  “I  have  come  to  the  conclusion  that  we 
as  educators  must  accept  change  in  order  to  implement  21st  century  teaching  so  that  our  students  become  21st  century 
learners.”  Finding  ways  to  elicit  that  change  through  well-planned  technology  integration  professional  development 
is  obstacle  that  can  be  overcome. 

To  better  address  technology  integration  professional  development  for  teachers,  one  should  have  a clear  picture 
about  teacher’s  attitudes  and  beliefs  towards  technology,  including  their  perceptions  about  how  technology  can  help 
them  to  become  better  teachers  and  how  technology  can  help  their  students.  Knowing  “the  motivation  to  learn 
depends  largely  on  the  learner’s  personality,  the  nature  of  the  thing  or  skill  to  be  learned,  and  the  learner’s 
perceptions  of  the  value  and  difficulty  of  learning  it”  (Keller,  1993,  p.  4),  considering  these  perceptions  is  critical  to 
the  successful  design  of  technology  integration  professional  development.  Keeping  in  mind  “the  most  important 
investment  a school  board,  administrators,  and  parents  can  make  in  a school  system  is  to  ensure  that  teachers 
continue  to  learn”  (American  Federation  of  Teachers,  n.d.),  taking  extra  steps  to  ensure  professional  development  is 
effective  is  well-worth  the  effort. 
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Abstract:  The  success  of  high  school  distance  education  in  Newfoundland  & 

Labrador,  hisotorically,  largely  depended  on  the  widely  known,  but  rarely 
documented  content-based  assistance  voluntarily  provided  by  from  school-based 
personnel.  Currently,  due  to  continuing  enrolment  decline,  more  rural  schools  find 
they  must  rely  upon  distance  education  to  offer  academic-level  courses  to  students 
with  a wide  range  of  abilities,  creating  new  responsibilities  for  teachers  that  have 
also  gone  undocumented.  This  study  will  document  the  duties  and  time  required  to 
provide  support  for  this  new  model  of  distance  education 

Newfoundland  and  Labrador  is  Canada’s  newest  province,1  the  second  smallest  in  terms  of  population  and 
perhaps  the  most  rural.  The  total  population  is  around  500,000,  the  majority  of  who  live  in  rural  areas. 
Newfoundland  and  Labrador  has  always  been  and  remains  a province  of  small  schools,  most  situated  in  rural  areas. 
Although  the  number  of  schools  over  all  has  declined  from  a high  of  1200  in  the  1960’s  to  less  than  300  today,  the 
scale  of  schooling  remains  small.  The  average  enrolment  of  all  schools  in  the  province  is  only  257  while  the  average 
for  the  179  rural  schools  is  163  students.  Ninety  of  the  smallest  of  these  rural  schools  are  all-  grade,  K-12  schools 
and  are  situated  in  the  more  remote  and  isolated  regions  of  the  province.  Close  to  25%  of  the  schools  have  less  than 
100  students  and  37  of  these  have  less  than  50  (Mulcahy,  2009). 

For  most  of  the  twentieth  century  educational  reformers  in  North  America  believed  the  problem  with  rural 
education  was  the  scale  of  schooling.  The  schools  were  too  small  to  provide  quality  education  for  rural  children  in  a 
cost  effective  manner.  The  solution  to  this  “problem”  was  obvious:  institute  a policy  of  closure  and  consolidation 
and  eliminate  as  many  small  schools  as  possible.  Rural  students  of  all  ages  would  be  transported  to  distant 
communities  where  they  could  enjoy  the  purported  educational  advantages  and  opportunities  only  available  in  larger 
schools. 

For  most  of  the  20th  century  educational  authorities  in  Canada  (and  the  United  States)  in  the  name  of 
educational  reform  have  pursued  a consistent  and  at  times  ruthless  policy  of  school  closure  and  consolidation. 
Nowhere  was  this  policy  more  “successful”  than  in  Newfoundland  and  Labrador.  Since  1966,  900  schools  mostly 
small  rural  community  schools  have  been  closed.  Parallel  to  the  closure  policy  was  an  extensive  use  of  bussing  that 
saw  increasing  numbers  of  students  bussed  for  longer  distances  to  larger  schools  further  and  further  away  from  their 
home  communities. 

By  the  end  of  the  20th  century  and  increasingly  in  the  21st  century  school  consolidation  had  run  its  course 
and  what  schools  that  could  be  closed  had  been  reformed  out  of  existence.  Increasingly,  the  small  rural  schools  that 
remained  relied  increasingly  on  various  forms  of  distance  education  to  provide  their  high  school  students  with 
access  to  the  programs  and  courses  they  need  to  graduate.  As  enrolment  continues  to  decline  and  site  based  teachers 
are  cut  from  schools  the  reliance  on  distance  learning  will  intensify. 

Distance  education,  in  a jurisdiction  like  Newfoundland  and  Labrador,  has  the  potential  to  make  the  size 
and  location  of  a rural  school  irrelevant  in  terms  of  its  ability  to  provide  students  access  via  the  Internet  to  any 
program  or  course  they  may  need  or  wish  to  take.  However,  certain  aspects  of  the  existing  model  in  this  province 
must  be  changed  if  all  students  are  to  have  an  equal  chance  of  succeeding  in  this  mode  of  learning. 


1 Newfoundland  and  Labrador  became  a Canadian  province  in  1949 
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Distance  Education  Overview 


The  problem  is  CDLI  is  being  used  at  our  school  to  teach  the  core  courses  ...  like  math  and  English 
and  all  the  sciences,  art  and  writing  ...  that’s  all  good  if  you  have  an  above  average  student  who  has  a 
great  work  ethic  but  if  you  have  a normal  student  who  is  easily  distracted  ...  sitting  him  in  front  of  a 
screen  is  not  going  to  work. . . and  we  are  going  to  lose  a lot  of  students  even  average  students...  their 
marks  are  going  down  ...  it  don’t  work  if  you  have  no  teachers  in  there  and  you  are  sitting  them  there 
in  front  of  a screen  and  they  are  supposed  to  work.  - parent  (Mulcahy,  Dibbon  & Norberg,  2008,  p. 

15) 

In  its  original  conception  in  Newfoundland  and  Labrador  in  1987,  distance  education  was  intended  to  provide 
supplementary  programming  to  small  rural  schools.  The  intended  cliental  were  the  top  academic  students  in  the 
school  who  were  interested  in  and  capable  of  working  in  a self-directed  and  independent  way. 

Since  1992,  with  the  province  experiencing  a dramatic  decline  in  student  enrolment  the  need  for  distance 
learning  has  intensified.  Hardest  hit  in  terms  of  enrolment  have  been  small  rural  schools.  As  a consequence  of  this 
decline  increasing  numbers  of  teaching  positions  have  been  eliminated  from  the  system.  The  ability  of  small  rural 
schools  to  offer  even  minimum  programs  and  courses  on  site  has  become  increasingly  problematic.  In  addition,  in 
recent  years  it  has  gotten  increasingly  difficult  to  staff  the  more  remote  and  isolated  small  schools  with  specialists  in 
the  areas  of  math,  science  and  foreign  language. 

The  Centre  for  Distance  Learning  and  Innovation  (CDLI)  was  established  (2000)  to  provide  high  school 
courses  to  small  rural  schools  via  the  Internet,  courses  not  available  on  site  because  of  the  schools  size  and/or 
location.  Each  year  since  2001  CDLI  has  increased  the  number  of  courses  available  online.  In  remote  and  isolated 
schools,  because  of  the  lack  of  curricular  options,  almost  all  students  must  take  one  or  more  distance  courses  in 
order  to  graduate. 

This  revised  mission  for  the  CDLI  is  a very  significant  change  in  the  purpose  for  CDLI  and  distance 
education  in  the  province.  Whereas  before  they  were  providing  a service  of  choice  for  an  elite  and  selective  group  of 
students,  they  are  now  operating  as  a necessary  mode  of  curriculum  delivery  for  even  the  most  academically 
challenged  and  immature  adolescents.  The  significance  of  this  cannot  be  overestimated:  students  must  master  the 
demands  of  distance  learning  or  compromise  their  future  life  chances. 

For  those  adolescents  who  possess  the  necessary  attributes2  to  succeed  in  this  virtual  environment,  CDLI 
provides  them  with  access  to  a wide  range  of  courses  that  they  would  not  otherwise  be  able  to  take.  For  those 
lacking  these  attributes,  online  learning  is  problematic. 

One  problem  that  participants  in  a recent  study  (Mulcahy  et.  al.,  2008)  identified  is  that  many  students  are 
just  not  suited  to  this  type  of  learning  because  they  are  unable  to  work  independently  without  direct  teacher 
supervision.  However,  as  smaller  schools  have  had  to  rely  more  heavily  on  CDLI,  all  students  have  to  take  one  of 
more  online  courses. 

Teachers  are  all  too  aware  of  the  problems  that  can  arise  from  leaving  adolescents  unsupervised.  However, 
their  on  site  responsibilities  leaves  little  time  to  monitor  distance  students.  One  teacher  explains  the  complexity  of 
the  situation,  “I  just  don’t  have  time  to  monitor  it.  I tried  but  I’d  be  leaving  my  own  class  to  check  on  them  and  that 
didn’t  seem  fair”  (Mulcahy  et.  al.,  2008,  p.  20).  Another  teacher  explains  what  can  happen  when  you  leave  students 
alone  in  a room,  “There  are  kids  fooling  around,  acting  up  and  distracting  others  and  no  one  is  going  to  be  the 
person  who  goes  to  the  principal”  (p.  20).  Students  themselves  report  many  distractions  with  online  learning,  “It’s 
just  harder  to  concentrate  when  you’re  on  the  computer.  There  are  too  many  temptations  to  go  to  MSN,  Facebook 
and  You  Tube”  (p.  20) 

Clearly  there  is  a need  for  more  supervision  and  support  for  students  engaged  in  distance  learning.  One 
parent  outlined  how  he  envisioned  the  role  of  a person  designated  to  work  with  distance  students.  He  felt  the  role  is 
more  than  just  supervising: 

Not  just  supervising  but  someone  who  meets  with  them  on  a regular  basis  to  ask  how  their  course  is 
going  ...  someone  to  say,  I notice  you  have  this  to  do  ...  you  have  an  assignment  due....  you  had  a 
test  last  week  and  you  didn’t  so  well  in  it ...  what’s  the  problem?  ...and  that  person  can  communicate 
with  the  CDLI  teacher  because  you  get  some  kids  who  don’t  ask  for  what  they  need..  Because  they 


2 A successful  online  learner  needs  to  be  disciplined  and  have  the  initiative  to  study  in  a flexible  environment 
without  the  constraints  and  supervision  of  a traditional  classroom.  Important  attributes  include:  the  ability  to  be 
organized,  motivated  and  self-directed.  Although  they  can  study  and  attend  class  at  their  leisure,  online  learners 
typically  do  not  procrastinate.  They  set  a pace  that  enables  them  to  comfortably  get  their  work  done.  They  are 
usually  very  committed  to  their  online  high  school  classes.  (Mulcahy  et  al,  2008,  p.  16) 
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don’t  know  what  they  need  and  to  expect  them  to  ask  someone  who  is  just  a voice  is  too  much  to 
expect  them  to  do  ...  that’s  an  additional  hurdle  to  them...  It’s  more  than  just  supervising...  (Mulcahy 
et.al.,  2008,  p.  20) 


The  Centre  for  Distance  Learning  and  Innovation 

The  method  of  delivery  utilized  by  the  CDLI  includes  both  synchronous  and  asynchronous  instruction.  The 
CDLI  provides  anywhere  from  30%  to  80%  of  the  students’  scheduled  time  (which  is  10  one-hour  periods  over  a 14 
day  cycle),  depending  on  the  subject  area,  in  synchronous  instruction  using  a virtual  classroom.  This  virtual 
classroom  allows  for  two-way  voice  over  the  Internet,  a shared,  interactive  whiteboard,  instant  messaging, 
application  sharing,  breakout  rooms,  and  interactive  quiz  and  survey  management  (Murphy  & Coffin,  2003). 
Through  this  virtual  classroom,  teachers  are  able  to  teach  in  much  the  same  way  they  would  in  a traditional 
classroom. 

The  asynchronous  instruction  is  conducted  using  a course  management  system  (CMS).  The  CMS  provides 
the  teacher  and  students  with  a variety  of  tools,  including  a discussion  forum,  a shared  calendar,  an  internal  e-mail 
system,  and  a place  to  house  the  course  web  pages.  The  course  web  pages  are  designed  by  a team  that  includes  a 
teacher  acting  as  a subject  matter  expert  and  a multimedia  specialist  to  add  images  and  interactive  items  into  the 
content.  The  course  web  pages  are  divided  into  the  units  called  for  in  the  curriculum  guide,  further  divided  into 
sections  which  are  akin  to  themes,  and  finally  into  lessons  which  can  be  completed  in  one  to  three  hours  of  student 
time.  Teachers  regularly  use  the  internal  e-mail  system  and  discussion  forums  to  communicate  with  their  students 
(Barbour,  2005). 

The  mediating  team  designed  to  support  this  delivery  at  the  school  level  is  officially  divided  into  three 
distinct  roles:  administrative,  technical,  and  coach.  The  administrative  m-teacher  often  proctors  tests  and  exams, 
monitors  student  attendance  and  behavior,  and  provides  for  scheduling,  enrollment  and  other  macro  level  items.  The 
technical  m-teacher  provides  initial  maintenance  and  trouble-shooting  for  the  CDLI  computers  (all  major  problems 
are  handled  by  a district-level  technician  or  by  the  CDLI).  Finally,  the  coach  m-teacher  provides  with  supervision 
and  support  (although  not  academic  support)  and  some  general  support  in  gaining  the  independent  learning  and  self- 
motivation  skills  that  may  be  needed  to  succeed  in  the  CDLI  environment  The  coach  may  also  be  used  to  proctor 
tests  and  exams,  along  with  monitoring  student  attendance  and  behavior.  Many  schools  will  assign  multiple  teachers 
to  perform  each  of  these  roles,  particularly  the  role  of  coach  (Barbour,  2007a). 

Background  on  the  Mediating  Teacher 
In  their  initial  evaluation  of  the  CDLI,  Coffin  and  Stevens  (2002)  indicated: 

e-learning  of  high  school  students  does  work.  With  the  present  level  of  connectivity  and  because  on-line 
high  school  programming  is  still  in  a nascent  stage  the  success  of  e-learning  will  be  restricted  to  students 
with  specific  traits,  habits  and  abilities”  (p.  49). 

The  difficulty  is  that,  as  Haughey  and  Muirhead  (1999)  described,  the  preferred  characteristics  of  students  involved 
in  K-12  online  learning  include  the  highly  motivated,  self-directed,  self-disciplined,  independent  learner  who  could 
read  and  write  well,  and  who  also  had  a strong  interest  in  or  ability  with  technology.  This  seems  to  justify  Mulcahy 
(2002)  concern  that  web-based  distance  education  may  not  be  accessible  be  all  students. 

The  problem  arises  when  more  rural  schools  are  forced  to  utilize  online  learning  to  offer  the  regular 
academic  curriculum  because  of  declining  student  populations.  Barbour  and  Mulcahy  (2008,  2009),  in  their 
examination  of  student  performance  in  CDLI  courses,  found  no  significant  differences  in  the  performance  of 
classroom-based  and  web-based  students.  However,  the  authors  have  questioned  whether  their  sample  of  web-based 
students  contains  students  from  the  same  range  of  abilities  as  would  be  found  in  the  classroom.  In  many  rural 
schools,  students  without  the  preferred  or  desired  characteristics  have  to  enroll  in  online  courses  in  order  to  graduate 
from  high  school  from  the  academic  program.  The  only  alternative  is  to  register  for  courses  in  the  basic  program, 
which  essentially  prevents  these  students  from  any  post-secondary  options  beyond  trade  school.  Recently,  in  their 
study  of  three  rural  schools  along  coastal  Labrador,  Mulcahy  et  al.  (2008)  found  that  two  of  the  schools  had  a higher 
percentage  of  students  enrolled  in  basic-level  courses  and  speculated  the  fact  that  many  of  the  academic  courses 
were  only  available  online  through  the  CDLI  as  a possible  cause.  An  initial  examination  of  this  issue  by  Mulcahy 
and  Barbour  (2009)  revealed  that  from  2006  to  2008  the  percentage  of  students  enrolled  in  basic-level  courses  was 
higher  in  CDLI  schools  than  in  schools  that  did  not  rely  upon  the  CDLI  for  its  academic  courses. 
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One  of  the  solutions  to  this  potential  problem,  and  one  of  the  suggestions  made  by  Mulcahy  et  al.  (2008)  to 
improve  upon  the  delivery  of  online  courses  by  the  CDLI,  was  “for  increased  support  and  supervision  in  the  school 
for  students  taking  distance  courses”  (p.  33).  As  Coffin  and  Stevens  (2002)  originally  noted  the  CDLI  “should  be 
treated  as  a significant  innovation  that  has  the  potential  to  transform  the  teacher-student  relationship  and,  hence,  the 
nature  of  learning  that  occurs.  Those  possibilities  need  to  be  explored  and  accommodations  made  to  online  programs 
and  the  delivery  mode”  (p.  24).  Cuban  (2002)  described  the  process  of  transforming  an  educational  system  as  first 
introducing  technology  and  then  “altering  fundamental  ways  of  achieve  organizational  goals. ...  [and]  introducing 
new  goals  and  interventions  that  transform  the  familiar  ways  of  doing  things”  (p.  229).  The  use  of  “volunteerism  and 
good  will  of  overworked  teachers”  is  consistent  with  the  former  model  of  distance  education  that  existed  in  the 
province  (Mulcahy  et  al.,  2008,  pp.  32-33).  Models  of  innovation  have  shown  us  that  change  will  only  come  when  it 
“offers  them  a better  way  to  do  something;  is  compatible  with  their  values,  beliefs  and  needs;  is  not  too  complex; 
can  be  tried  out  before  adoption;  and  has  observable  benefits”  (Surrey  & Ely,  2007,  p.  106).  Clearly  more  research  is 
needed  into  the  role,  responsibilities,  and  time  commitment  of  the  in-school  teachers  who  have  volunteered  their 
time  and  good  will  to  date. 

Unfortunately,  there  has  been  little  research  on  how  these  school-based  responsibilities  are  actually  being 
implemented.  Mulcahy  (2002)  was  concerned  that  with  less  student  selectivity,  an  increasing  need  for  distance 
education  by  rural  schools,  and  the  nature  of  the  proposed  delivery,  teachers  who  were  given  mediating 
responsibilities  for  the  CDLI  would  play  a critical  role  in  the  success  of  this  new  initiative,  and  that  there  was  a 
failure  to  consider  the  additional  workload  that  would  be  placed  on  these  rural  teachers.  This  concern  was  well 
founded,  at  least  during  the  initial  year  of  the  CDLI.  Barbour  and  Mulcahy  (2004)  found  teachers  in  one  district 
reported  “quite  a burden  [was]  placed  upon  them  due  to  the  wide  range  of  duties  and  time  commitment  associated 
with  these  new  responsibilities”  (Conclusion  section,  ^]  2).  They  also  found  these  teachers  provided  technical  and 
instruction  assistance,  both  of  which  were  outside  of  the  original  vision  of  the  ministerial  panel.  In  a similar 
examination  of  school-based  teachers  in  the  ACCESS  Alabama  program,  Roblyer,  Freeman,  Stabler  and 
Schneidmiller  (2007)  found  that  “facilitators  that  are  directly  working  with  students  day  by  day  are  key  to  the 
success  of  the  [K-12  online  learning]  program”  (p.  11).  found  Unfortunately,  no  further  research  exploring  how 
these  additional  responsibilities  are  managed  at  the  school  level  has  been  conducted. 

Methodology 

To  begin  this  study  we  sought  permission  from  each  of  the  five  school  districts  in  Newfoundland  and 
Labrador  for  permission  to  conduct  research  in  their  district.  This  was  followed  by  contacting  individual  school 
principals,  and  then  m-team  members,  to  seek  their  participation  in  this  research  study.  As  the  researchers  were 
engaged  in  a separate  research  project  with  District  1,  the  Director  of  this  district  declined  participation.  In  addition, 
some  principals  in  the  remaining  four  districts  also  declined  participation.  This  resulted  in  the  follow  levels  of 
participation. 


Table  1.  Level  of  Participation  from  Schools  who  Access  the  CDLI 


Sch 

ools 

M-Team 

Members 

Schools 

Participating 

M-Team 

Members 

Participating 

Completed  Opted-Out 

Not  responsive 

District  1 

10 

34 

- 

District  2 

32 

83  18 

46  16 

7 

23 

District  3 

33 

102  33 

102  17 

27 

58 

District  4 

21 

62  20 

58  15 

9 

34 

District  5 

2 

3 2 

3 0 

1 

2 

Private 

2 

5 2 

5 1 

0 

4 

Schools 

Totals 

100 

289  7 

5 

214  4 

9 

44 

121 

Even  without  the  participation  of  District  1,  the  possible  sample  represented  75%  of  the  schools  participating  in  the 
CDLI  and  74%  of  the  potential  m-teachers.  We  had  a 23%  response  rate,  with  21%  of  m-teachers  opted  out  of  the 
study  and  56%  who  were  unresponsive. 
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The  initial  method  of  data  collection  was  a revised  version  of  the  survey  utilized  by  Babour  and  Mulcahy 
(2004),  only  adding  questions  to  volunteer  for  future  data  collection.  Surveys  are  useful  for  providing  structured 
responses  to  many  questions  (Yin,  2003),  and  can  also  be  useful  tools  in  obtaining  data  about  characteristics, 
attitudes,  and  beliefs  about  the  participants,  and  for  making  inferences  about  a group  of  participants  (Marshall  & 
Rossman,  1999).  In  addition  to  the  surveys,  we  also  had  six  volunteers  complete  weekly  logs  of  the  duties  that  they 
undertook  in  relation  to  the  CDLI  or  students  enrolled  in  CDL1  courses  and  the  amount  of  time  it  took  to  complete 
these  duties.  Often  what  people  say  they  have  done  in  retrospect  differ  from  what  they  actually  did  in  the  moment 
(Rathje  & Murphy,  1992),  so  a written  account  that  was  relatively  closer  to  the  events  in  terms  of  time  may  have 
provided  a more  accurate  description  of  the  events. 

Finally,  we  conducted  seven  semi-structured  interviews  that  are  currently  being  transcribed.  Semi- 
structured  interviews  allow  discussion  of  a greater  breadth  of  topics  than  more  structured  interviews  (Fontana  & 
Frey,  2000),  and  also  allow  for  a natural  flow  to  the  interaction  (Patton,  2002).  This  article  reports  on  the  data 
collected  from  the  surveys  and  weekly  logs. 


Results  and  Discussion 

The  initial  questions  on  the  survey  explored  the  list  of  activities  outlined  in  a 2001  memo  sent  by  the 
Director  of  the  CDLI  to  all  participating  schools  and  school  districts  outlining  the  duties  that  should  and  should  not 
be  performed  by  m-teachers.  Table  2 provides  the  percentage  of  m-teachers  who  selected  each  level,  along  with  the 
results  of  found  by  Barbour  and  Mulcahy  (2004)  - in  grey  - for  comparison. 

In  the  results  from  this  study,  there  were  a high  percentage  of  m-teachers  who  selected  “sometimes”  or 
“often”  for  both  the  monitoring  the  progress  of  distance  learning  students  and  the  following-up  with  such  students  to 
ensure  future  compliance  statements.  These  two  statements  also  saw  a significant  increase  from  the  results  found  in 
2001-02,  possibly  due  to  the  fact  that  in  the  CDLI  initial  year  of  operation  many  principals  were  advised  to  act  as 
gatekeepers  for  the  online  courses  that  were  being  piloted.  As  students  with  a wider  range  of  abilities  are  enrolling  in 
CDLI  courses  m-teachers  have  to  spend  more  time  monitoring  their  progress  and  follow-up  with  these  academically 
weaker  students. 


Table  2.  M-Teachers  Feedback  of  Various  Duties  (current  study  and  2001-02) 


Neve 

r 

Sometimes 

Often 

Supervising  distance  learning  students  while  they  engage  in  online 

17  54 

29 

activities. 

0 

80 

20 

Monitoring  the  progress  of  distance  learning  students,  including 
accepting  e-mail  notification  from  the  e-teacher  which  express  concern 

8 31 

60 

regarding  the  failure  of  a student  to  submit  assignments,  exams,  etc.  on 
time. 

0 

80 

20 

Following-up  with  such  students  to  ensure  future  compliance. 

13 

44 

44 

0 

80 

20 

Accepting  grades  and  reports  from  the  e-teacher  and  ensure  that  these  get 

17  17 

67 

entered  in  the  students  term/end  of  year  report  cards. 

0 

60 

40 

Providing  limited  assistance  to  students  who  encounter  difficulty  in  using 
asynchronous  communication  tools  (chat,  discussion  threads,  e-mail,  etc., 

27  56 

17 

web  browser,  and  learning  management  system). 

20 

80 

0 

Including  online  students  on  the  teacher's  class  list  and  as  such  follow-up 
on  absences  from  class  as  would  be  the  case  with  other  students  in  that 

38  38 

25 

class  whom  the  m-teacher  instructs  directly. 

20 

40 

40 

Meeting,  as  requested,  with  the  e-teacher,  web-based  initiatives 

50  44 

6 

facilitator,  high  school  program  specialist. 

0 

80 

20 

In  consultation  with  the  CDLI,  Virtual  Teachers  Centre  of  the  NLTA  and 
the  School  District,  assisting  staff  colleagues  in  acquiring  skills 
necessary  for  accessing  web-based  professional  development 
opportunities. 

65  35 

0 

40 

60 

0 

Participating  in  district  course  implementation  team  meetings  upon 

79  15 

6 

request. 

60 

40 

0 

Participating  in  provincial  in-services  and  forums  upon  request. 

60 

33 

6 

0 

100 

0 

20 


When  asked  about  other  duties  that  m-teachers  were  known  to  perform  that  were  not  necessarily  outlined  in  the 
original  2001  memo,  the  following  results  were  found. 


Table  3.  Time  Spent  per  Week  on  M-Teacher  Duties 


Duty 

# of 

m-teachers 

Time  spent 
/week 

Adjusted 
time/week 
based  on  % of 
m-teachers 

Supervising  tests/exams 

40 

58  minutes 

48  minutes 

Tracking  down  missing  assignments/homework 

20 

32  minutes 

13  minutes 

Recording  attendance  or  other  administrative  data 

19 

18  minutes 

7 minutes 

Providing  content-based  tutoring 

20 

42  minutes 

17  minutes 

Providing  technology-based  tutoring 

23 

24  minutes 

11  minutes 

Providing  technical  trouble-shooting 

34 

36  minutes 

25  minutes 

[Other]  Setting  up  and/or  supervising  science  labs 

8 

60  minutes 

1 0 minutes 

[Other]  Preparing  and  submitting  tests/exams 

9 

18  minutes 

3 minutes 

[Other]  Encouraging  students  to  attend  class 

1 

60  minutes 

1 minutes 

[Other]  Communication  with  e-teacher  or  CDLI 

1 

60  minutes 

1 minutes 

[Other]  Downloading  programs 

1 

20  minutes 

1 minutes 

The  time  adjusted  time  spent  per  week  based  upon  the  percentage  of  m-teachers  who  selected  who  selected  each  of 
the  duties  indicates  that  m-teachers  spent  an  average  of  139  minutes/week  or  2.3  hours/week  engaged  in  duties 
related  to  their  role  as  a CDLI  m-teacher.  The  timetable  used  by  the  CDLI  is  a 14x5  schedule  (i.e.,  five  one  hour 
classes  a day  over  a 14  day  cycle),  which  allows  for  ten  classes  from  each  of  seven  courses  or  slots.  A typical 
teacher  in  Newfoundland  and  Labrador  would  teach  six  out  of  the  seven  slots.  An  average  of  2.3  hours/week  equates 
to  6.5  hours/14-day  cycle  or  approximately  two  thirds  of  a full  course  slot. 

The  information  provided  by  the  six  teachers  who  completed  the  weekly  logs  over  a four  week  period 
found  that  these  teachers  were  primarily  occupied  by  the  following  tasks:  supervising  tests/exams/class,  tracking 
down  missing  assignments/homework,  recording  attendance  or  other  administrative  data,  providing  content-based 
tutoring,  providing  technology-based  tutoring,  providing  technical  trouble  shooting,  supervising  science  labs,  and 
preparing  and  submitting  tests/exams.  It  should  be  noted  that  the  total  time  reported  on  these  weekly  logs  was 
approximately  half  the  minutes/week  that  was  reported  in  the  surveys  (i.e.,  148  minutes/week  vs.  286 
minutes/week).  This  result  may  be  due  to  the  specific  six  individuals  who  completed  the  weekly  logs  (i.e.,  these 
individuals  may  not  have  been  as  busy  as  their  fellow  m-teachers  at  their  own  or  other  schools.  It  may  have  also 
been  due  to  the  time  of  year  these  weekly  logs  were  completed  (i.e.,  February-March  may  have  been  lighter  months 
in  terms  of  CDLI  responsibilities).  Either  way,  the  total  amount  of  the  time  reported  in  the  logs  was  still  more  than 
the  amount  reported  by  Barbour  and  Mulcahy  (2004). 

Based  upon  these  results,  m-teachers  potentially  lose  up  to  approximately  two  thirds  of  their  preparation 
time  (or  would  have  significant  portions  of  time  when  they  are  scheduled  to  teach  other  students  in  the  school) 
occupied  with  CDLI  duties.  Barbour  and  Mulcahy  (2004)  found  that  m-teachers  spent  105  minutes/week  or 
approximately  5 hours/14-day  cycle  engaged  in  their  CDLI  duties.  This  represents  an  increase  of  1.5  hours/cycle, 
which  can  also  likely  be  attributed  to  the  increase  in  students  who  do  not  exhibit  the  characteristics  outlined  earlier 
by  Flaughey  and  Murihead  (1999). 


Policy  Change  Needed 

The  existing  policy  of  CDLI  maintains  that  students  taking  distance  courses  do  not  need  academic  support 
and  supervision  from  school  based  teachers.  They,  purportedly,  receive  all  the  academic  support  they  need  from 
their  online  instructors  and  other  support  put  into  place  by  the  CDLI  (Barbour,  2007b). 

Flowever,  as  the  range  of  students  enrolling  in  distance  courses  has  expanded  it  has  become  clear  that  some 
students  need  a great  deal  more  school-based  support  and  supervision  than  that  envisioned  in  the  current  CDLI 
model.  Many  of  these  students  lack  both  motivation  and  self-direction.  Left  unsupervised  they  will  spend  their  time 
chatting  with  friends  on  MSN  or  Facebook,  downloading  music  and  playing  computer  games  rather  than  attending 
to  their  online  lessons.  Many  others  are  lacking  in  academic  ability  and  find  it  very  difficult  to  complete  their  work 
without  teacher  support.  While  there  is  academic  tutoring  available  online  many  lack  the  confidence  to  initiate 
contact  with  their  e-teachers  or  e-tutors. 
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The  few  teachers  working  on  site  in  the  remote  and  isolated  schools  do  as  much  as  they  can  for  these 
students  but  their  school-based  workload  is  intense.  In  some  situations  teachers  have  to  take  time  away  from  their 
existing  teaching  duties  in  order  to  assist  an  online  student.  This  means  they  have  to  interrupt  and  disrupt  their 
school  based  classes  to  provide  the  needed  assistance  (Barbour,  2007b).  This  is  unfair  to  the  school  based  students. 

In  other  situations  the  help  has  to  be  provided  at  recess,  lunch  time  or  after  school.  Rural  teachers  have  to  volunteer 
their  precious  free  time  to  work  with  online  students.  Neither  of  these  situations  is  pedagogically  sound  or 
acceptable.  Distance  students  cannot  be  dependent  on  the  volunteerism  and  good  will  of  overworked  rural  teachers. 
Nor  can  their  educational  needs  be  defined  as  an  additional  assignment  for  a teacher  who  already  has  a full  and 
demanding  teaching  load. 

A modification  to  the  existing  CDL1  model  is  appropriate  and  needed.  There  especially  has  to  be  some 
provision  for  increased  support  and  supervision  in  the  school  for  students  taking  distance  courses.  There  is  a need 
for  someone  who  would  have  dedicated  time  away  from  other  duties  and  responsibilities  to  work  with  a school’s 
CDLI  students.  This  support  person  would  not  be  primarily  academic.  CDL1  has  excellent  teachers  who  make 
themselves,  for  the  most  part,  readily  available  to  their  students.  The  academic  expertise  is  in  place.  The  real  need  is 
for  a person  who  would  monitor  and  supervise  students  while  they  engage  in  their  online  courses.  This  person  would 
act  as  a liaison  between  the  student,  the  online  teacher,  the  parents  and  the  school’s  personnel.  They  would  help 
students  keep  track  of  their  assignments  and  tests,  assist  with  technical  problems  and  facilitate  contact  between 
students  and  their  online  instructor  (Mulcahy,  2002). 

In  a recent  study  of  teacher  work  load  and  allocation  in  the  province,  Shortall  and  Greene-Fraize  (2007) 
recommended  allocating  one  specialist  (i.e.,  teaching  professional)  to  schools  ‘‘at  a rate  of  one  per  175  students  to 
support  the  areas  of. . . Centre  for  Distance  Learning  and  Innovation  (CDLI)  support”  (p.  109).  Later  in  that  report,  in 
a section  entitled  “Other  Issues”,  the  authors  recognized  that  “CDLI  courses  need  school  personnel  support  for 
administration,  supervision,  and  subject  support”  (emphasis  added,  p.  180),  and  recommended  that  the  Government 
consider  this  issue  when  considering  future  plans  for  growth  and  development.  This  change  in  policy,  if  the 
Government  accepted  it,  would  ameliorate  many  of  the  serious  concerns  raised  by  rural  teachers,  parents  and 
students.  There  is  no  question  that  more  and  more  students  are  going  to  have  to  rely  on  distance  education  provided 
by  CDLI  simply  to  graduate  from  high  school.  If  students  are  going  to  graduate  with  an  academic  qualification  (i.e., 
the  ability  to  attend  college  or  university),  depending  on  the  size  of  the  school  they  attend,  most  will  have  to  take  a 
significant  portion  of  their  program  online.  If  the  right  supports  are  in  place  more  students  will  avail  of  the  academic 
programs  and,  more  importantly,  have  a greater  opportunity  to  succeed. 
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Abstract.  Th  is  p aper  fo  cuses  o n exp  eriences  o f un  dergraduate  Filip  ino  stud  ents  with  o nline 
technology  from  a Man  ila-based  university  (Philippines)  resulting  from  using  Moodle  (Modular 
Object-Oriented  Dynamic  Learning  Environment)  to  collaborate  with  a group  of  students  located 
in  southern  United  States  (Georgia).  It  di  scusses  the  de  sign  and  de  velopment  of  a virtual  space 
using  Moodle  to  facilitate  an  educational  experience  for  in  dividuals  separated  by  great  d istance 
and  variance  in  contexts.  Also,  it  discusses  the  perceived  outcomes  of  collaborating  with  another 
group  in  completing  a common  task.  Further,  it  shares  insights  and  implications  in  designing  and 
developing  online  experiences  for  non-U.  S.  participants  based  on  reported  experiences  of  Filipino 
undergraduate  st  udents.  Fi  nally,  t he  pa  per  a ddresses  t he  im  pact  of  online  c ross-cultural 
experiences  on  student  learning. 


Introduction 

The  continuous  introduction  of  new  information  and  communication  technologies  that  support  teaching  and 
learning  i n hi  gher  e ducation  has  presented  new  chal  lenges  t o both  fac  ulty  and  students.  T hese  t echnologies  a re 
transforming  higher  education  at  a very  rapid  rate  that  make  everyone  ask  with  questions  like:  What  are  some  of  the 
new  t echniques  I nee  d t o know  t o us  e t hese  t ools  e ffectively?  How  effective  a re  t hese  to  ols  in  th  e d elivery  o f 
university-oriented  curriculum?  How  does  this  influence  or  shape  the  development  and  delivery  of  courses  in  higher 
education  worldwide  ? More  im  portantly,  faculty  and  students  are  now  asking  how  their  teaching  a nd  lea  rning 
experiences  can  b e enh  anced  by  wo  rking  together  w ith  their  co  unterparts  in  sep  arate  lo  cations  ar  ound  th  e wo  rid 
through  these  technologies. 

The  promise  of  a digital  world  ha  s brought  a bout  i ncreased  i nterests  fr  om  diffe  rent  stakeholders  — 
administrators,  faculty  m embers,  a nd  students  — to  ta  ke  on  the  c hallenges  of  onli  ne  teachi  ng  a nd  learni  ng. 
Moreover,  the  continuous  introduction  of  instructional  technologies  in  higher  education  has  led  to  experimentation 
on  what  works  online  for  different  populations  and  contexts  (Brett  & Nag  ra,  2005).  Researchers  and  p ractitioners 
alike  h ave  written  articles  s upporting  th  e benefits  o f distance  learn  ing  (Garrison  , 2000;  Sch  wartzman,  20  07). 
Students  engaged  in  distance  learning  experiences  gain  a global  dimension  and  acquire  a competitive  advantage  on 
perspective  t owards  real  life  situ  ations  (Cassano  , 2008).  With  distance  learning  experiences,  s tudents  h ave 
opportunities  to  bring  classrooms  to  gether  fro  m aero  ss  cultures  and  d istances  with  un  ique  and  ch  allenging 
perspectives. 
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This  case  st  udy  involved  two  fac  ulty  members  who  were  teaching  unde  rgraduate  courses  for  pre-service 
teachers,  and  agreed  to  investigate  the  impact  of  collaborative  online  experiences  among  their  students  through  the 
use  of  technology  and  by  designing  common  activities  and  assignment  for  students  to  complete.  In  addition  to  the 
technological  and  c urricular  chal  lenges,  differences  i n learner  c haracteristics  and  cultural  back  ground  w ere 
considered. 

Contexts  and  Participants 

An  online  learning  environment  was  designed  and  developed  to  host  the  communication,  connection, 
collaboration,  and  community-building  activities  of  two  groups  of  undergraduate  students  in  two  distant  locations: 
Manila,  Philippines  and  Georgia,  United  States.  Although  separated  by  great  distance,  these  groups  of  undergraduate 
students  worked  together  in  completing  their  respective  assignment  of  developing  a K-12  teaching  (instructional) 
unit  by  sharing  information  relevant  to  completing  the  task. 

Initially,  the  U.S.  faculty  member’s  interest  in  getting  his  students  to  participate  in  this  project  was  the  need 
for  exposure  to  diverse  perspectives.  The  university’s  location  in  a rural  part  of  southern  United  States  limits 
exposure  and  interaction  with  diverse  populations.  For  the  Philippine-based  faculty  member,  she  was  interested  in 
learning  more  how  to  use  online  technology  like  Moodle  as  well  as  exposing  her  students  to  a technology-rich 
learning  environment. 

This  paper  focuses  only  on  the  Filipino  undergraduate  students’  experiences.  References  to  the  American 
students  and  their  experiences  are  limited  except  when  providing  context  to  facilitate  a b etter  understanding  of  the 
process  that  occurred  between  the  two  groups.  The  Filipino  students  were  enrolled  in  an  undergraduate  course  that 
focused  o n t he  use  oft  echnology  i n pre-school  e ducation  fr  om  a M anila-based  university.  Due  t o di  fferences  i n 
academic  calendars  between  the  Philippine  s and  the  United  States,  the  Filipino  students  started  in  November  and 
completed  their  term  in  mid-March  of  the  following  year.  The  American  students  started  their  term  in  early  January 
of  the  following  year,  were  enrolled  in  undergraduate  technology  course,  and  finished  their  course  activities  in  early 
May.  Initially,  there  were  a total  o f 14  Filipino  students  enrolled.  Data  from  1 3 students  were  used  for  this  research 
project  due  to  non-completion  of  online  activities  by  one  student.  All  Filipino  students  were  female  and  majoring  in 
early  childhood  education. 

Learner  Characteristics 

The  literature  id  entifies  leam  er  eh  aracteristics  as  an  i nfluencing  facto  r in  eng  aging  in  edu  cational 
processes.  Th  e research  ers  used  th  e research  on  personality  assess  ment  b y Go  ldberg  ( 1 99  2)  an  d co  lleagues  th  at 
identified  five  basic  factors:  Extroversion,  Openness,  Agreeab  leness,  Conscientiousness , and  Em  otional  Stability. 
Extroversion,  in  comparison  to  introversion,  is  the  "people  person"  factor.  It  is  primarily  focused  on  behaviors  that 
manifest  ou  tgoingness  an  d so  ciability.  Also  , it  is  asso  ciated  with  in  dependence  and  decisiveness.  Openness 
(sometimes  called  Openness  to  Experience)  looks  at  being  open-minded,  curious,  creative,  artistic,  and  adventurous. 

A high  score  in  the  Agreea  bleness  scale  can  be  in  terpreted  th  at  a person  is  id  entified  as  coop  erative, 
considerate,  good-natured,  and  t olerant.  Thi  s coul  d m ean  t hat  t hese  i ndividuals  m ay  avoi  d co  nflict. 
Conscientiousness  examines  whether  one  is  organized,  schedule-oriented,  accurate,  extremely  careful,  disciplined, 
and  shows  ethical  behavior  and  integrity.  Emotional  Stability  (vs.  Neuroticism)  considers  how  a person  responds  to 
stress,  anxiety,  and  depression. 


Table  1 

Pattern  of  Student  Responses  to  Personality  Inventory  on  Extraversion  and  Openness  Scale  (N  = 13) 


Student  Response 

Extraversion  (%) 

/ 

Openness  (%) 

/ 

Extremely 

0.8  1 

0.0 

0 

Highly 

69.2  9 

77.0 

10 

Moderately 

23.0  3 

23.0 

3 

Mildly 

0.0  0 

0.0 

0 

Barely  or  not  at  all 

0.0  0 

0.0 

0 

Total  % 

100.0  13 

100.0 

13 
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Results  of  the  personality  assessments  for  the  Filipino  students  reported  moderate  to  high  (92%)  in  the 
extraversion  scale.  Also,  all  students  reported  moderate  to  high  (100%)  in  the  openness  scale.  One  interpretation  of 
these  results  could  be  that  this  particular  group  of  students  easily  make  friends  and  welcome  new  experiences.  These 
characteristics  seem  important  for  those  who  are  trying  something  different  and  for  the  first  time  as  part  of  the 
academic  experience. 


Table  2 

Pattern  of  Student  Responses  to  Personality  Inventory  on 
Agreeableness,  Conscientiousness,  and  Emotional  Stability  Scale  (N  = 13) 


Student  Response 

Agreeableness 

(%) 

/ 

Conscientiousness 

(%) 

/ 

Emotional  Stability 
(%) 

/ 

Always 

30.8  4 

15.4 

2 

0.0 

0 

Often 

61.5  8 

38.5 

5 

15.4 

2 

Usually 

7.7  1 

46.1 

6 

69.2 

9 

Seldom 

0.0  0 

0.0 

0 

15.4 

2 

Hardly 

0.0  0 

0.0 

0 

0.0 

0 

Total  % 

100.0 

13 

100.0 

13  1 

) 0.0  13 

Results  on  the  factors  were  the  following:  12  students  reported  high  on  Agreeableness,  i.e.,  often  to  always 
(92%);  11  reported  high  on  Conscientiousness,  i.e.,  "usually"  to  "often"  (85%);  and  9 reported  average  on  Emotional 
Stability,  i.e.,  usually  (69%).  Interpretation  of  these  results  could  mean  that  this  group  can  be  characterized  as 
tolerant  for  extreme  changes  in  their  environment  especially  with  conflicts.  Another  interpretation  suggests  that  this 
group  is  focused  on  the  task  at  hand.  Finally,  it  seems  that  this  group  tend  to  manage  stressful  situations  very  well, 

i.e.,  demands  for  their  time  and  attention  in  completing  the  various  tasks  in  the  course. 

Moodle  as  a Learning  Space 

The  collaborative  virtual  space  for  the  two  groups  to  interact  was  supported  by  Moodle,  a server-based  and 
open  source  application  that  functioned  as  a 1 earning  management  system  ( Moodle , 2008).  It  was  designed  as  an 
environment  where  students  interacted  with  each  other  while  completing  the  assigned  task.  Moodle  was  selected  for 
low/no  cost,  ease  of  set-up  and  m anagement,  user-friendliness,  and  server-based  functions  sim ilar  to  WebCT  or 
Blackboard  (B  lack.  Beck,  Dawson,  Ji  nks,  & DiPietro,  2007).  Moodle  features  al  low  i nstructors  an  d st  udents  t o 
interact  whe  n sharing  i deas,  bui  lding  co  ncepts,  re  fleeting  o n ex  periences,  an  d co  nstructing  k nowledge  (Corich, 
2005).  In  using  Moodle  as  veh  icle  for  online  collaboration,  both  groups  of  students  were  required  to  co  mplete  the 
same  assignment  (i.e.,  d eve  lop  a K-12  instructional  unit  collaboratively),  participate  in  threaded  discussions,  and 
submit  reflection  postings/papers. 


Research  Questions 

For  this  study,  the  researchers  were  in  terested  in  the  experiences  of  Filipino  undergraduate  students  who 
participated  in  an  online  co  llaboration  wit  h American  st  udents.  Specifically,  the  res  earchers  were  in  terested  in 
factors  that  support  online  cross-cultural  exchanges,  such  as,  learner  characteristics  (Santiago  & Nakayama,  2006) 
and  others  (Tyler  & Baylen,  1998).  Also,  the  researchers  were  in  terested  in  how  Filipino  undergraduate  students 
perceive  and  value  their  experiences  of  using  technology  to  enhance  knowledge  and  be  successful  in  completing  the 
given  learning  tasks. 

1.  How  do  the  Filipino  students  perceive  their  online  experiences?  What  are  the  similarities  and  differences  of 
perceptions  between  the  Filipino  students  and  their  American  counterparts? 

2.  What  component  of  the  online  experience  was  valued  by  the  Filipino  students?  What  are  the  similarities  and 
differences  of  valued  experiences  between  the  Filipino  students  and  their  American  counterparts? 

3.  How  can  technology  support  similar  online  cross-cultural  exchanges  that  enhance  undergraduate  students' 
educational  experiences  from  other  countries? 
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Analysis  of  Data 


Narrative  texts  from  postings  in  online  discussions  and  responses  to  prompt  questions  were  analyzed  for 
common  and  recurring  themes  and  patterns.  The  researchers  reviewed  the  results  vis-a-vis  current  research  literature 
on  online  teaching  and  learning  as  well  as  international  issues  related  to  using  technology-based  tools.  The  analysis 
of  completed  tasks  by  the  undergraduate  Filipino  students  focused  on  the  quality  of  technology  integration  ideas 
shared,  creative  planning  demonstrated,  and  types  of  support  received  from  American  students. 

Findings  and  Discussion 

Researchers  focused  on  the  discussion  of  findings  on  the  following  — 1 ) perceptions  of  online  experience  as 
well  as  similarities  and  differences  of  perceptions  between  the  two  groups,  2)  valued  online  experiences  as  well  as 
similarities  and  differences  in  what  was  valued  between  the  two  groups,  and  3)  technology -based  tools  and 
experiences  that  supported  learning  among  groups  with  varied  cultural  backgrounds. 

Perceptions  of  Online  Experiences 

On  perceptions  of  the  online  experience,  the  content  analysis  of  reflection  postings  and  papers  revealed 
four  online  experiences  that  were  identified  as  significant: 

1)  being  able  to  share  ideas  about  teaching; 

2)  receiving  feedback  and/or  comment  on  the  K-12  instructional  unit; 

3)  interacting  with  students  from  another  country;  and 

4)  realizing  that  future  teachers  (even  across  cultures)  have  similar  concerns.  The  Filipino  group  identified  an 

additional  significant  experience,  i.e.,  meeting  new  friends. 

Both  groups  indicated  that  these  experiences  would  not  have  been  possible  without  the  virtual  space  that  Moodle 
provided. 

Valuing  Online  Experiences 

Analysis  of  qualitative  data  from  students  identified  communication,  connection  and  collaboration  as 
valued  online  learning  experiences.  Researchers  defined  communication,  in  this  context,  as  an  individual's  ability  to 
send  messages  using  a technology-based  tool.  Students  expressed  that  emails  and  threaded  discussions  enhanced 
their  online  learning  experience. 

Connection,  as  defined  by  the  researchers,  is  the  ability  to  link  or  relate  the  messages  shared  online  to 
one's  past,  current  and  future  life  experiences.  There  was  strong  evidence  that  when  sharing  what  they  have  seen  and 
experienced  in  pre-K  classrooms  through  email  messages  or  threaded  discussions,  students  created  that  connection 
to  individual  experiences  in  relation  to  self,  others,  community  or  the  world. 

Finally,  collaboration,  based  on  the  researchers’  definition,  is  the  ability  to  work  with  others  in  an  online 
environment  and  support  each  other  in  completing  a task.  Students  were  able  to  give  feedback  and  comments,  thus 
contributing  mutually  to  the  development  of  improved  K-12  instructional  units  and  to  this  online  experience 
becoming  a foundation  for  future  experiences. 

Technology  Across  Cultures 

It  is  possible  for  learners  from  two  geographical  locations  to  successfully  work  and  learn  using  available 
technology.  Technology -based  activities  that  facilitate  students’  ability  to  communicate,  connect,  and  collaborate 
can  enhance  students’  online  experiences.  Also,  the  opportunity  to  interact  with  peers  from  another  culture 
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facilitated  the  acquisition  of  new  understanding  on  how  the  similarities  and  differences  in  needs,  views  or 
perspectives  on  issues  or  concerns  that  involve  work  with  children. 

Implications  of  the  Study 

The  results  of  this  research  project  shed  light  to  various  factors  that  instructional  designers  and  developers 
need  to  consider  when  working  with  non-U. S.  population.  One  factor  that  needs  to  be  considered  is  the  integration 
of  online  collaborative  activities  that  facilitate  d evelopment  of  relationship  among  participants.  It  is  recommended 
that  key  activities  facilitating  communication,  collaboration,  and  community  building  need  to  be  in  place  as  part  of 
the  course  design  to  continuously  engage  students  in  small  groups  and  as  a class. 

The  researchers  suggest  taking  a second  look  at  learner  characteristics  because  they  can  play  a major  role  in 
impacting  student  engagement  in  completing  online  tasks.  Further,  it  is  critical  to  provide  activities  where  students 
make  co  nnections  between  t heir  e xperiences  i n an  d out  o f t he  class  room  to  what  is  ha  ppening  i n t heir  online 
environments.  Tasks  need  to  be  neutral  and  designed  to  facilitate  sharing  information  and  ideas  instead  of  tackling 
controversial  issues. 

Finally,  in  creating  a positive  learning  environment  for  the  students,  it  is  important  to  provide  opportunities 
for  effect  ive  pi  anning,  b uilding  use  r-friendly  i nterfaces,  and  identifying  alterna  tive  com  munication  t ools  an  d 
strategies.  If  all  of  these  are  considered,  then  the  journey  to  a positive,  active,  online  collaborative  experience  among 
students  will  be  realized  whether  they  are  from  the  United  States  or  from  abroad. 

Conclusion 

This  paper  focuses  on  further  dissemination  of  information  on  how  Moodle  supported  the  collaboration  of 
two  groups  separated  by  great  distance  and  variance  in  contexts.  The  discussion  of  this  paper  focuses  on  the 
experiences  on  Filipino  undergraduate  students  - their  perceptions  and  the  things  they  valued  from  the  online 
experience  and  their  use  of  Moodle,  an  open  source  web-based  application. 

Insights  gained  from  this  study  focused  on  how  technology  can  support  similar  online  cross-cultural  and 
educational  exchanges.  Lessons  learned  focused  on  designing  online  experiences  to  enhance  undergraduate 
experience  might  need  to  include  more  activities  facilitating  communication,  collaboration  and  community  building 
for  non-U. S.  student  populations  in  order  to  development  relationships  among  students  from  both  groups.  Finally, 
the  researchers  conclude  that  positive  educational  experiences  can  be  supported  by  using  a learning  management 
system  like  Moodle  with  non-U. S.  students. 

Given  the  small  number  of  Filipino  undergraduate  students  in  this  study,  the  results  serve  as  guide  in 
creating  a more  rigorous  design  of  investigating  non-U.  S.  population’s  experiences  with  online  learning. 
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Abstract 

The  purpose  of  this  research  was  to  determine  the  effect  of  learning  games  on  preschool  children  in  terms 
of  learning  outcomes,  level  of  engagement  and  satisfaction.  The  presenters  will  discuss  the  preliminary  findings  of 
this  research  as  well  as  the  practical  implications  of  using  learning  games  with  young  children.  Additionally,  the 
presenters  will  discuss  the  teacher  satisfaction  with  this  intervention  as  well  as  future  research  opportunities  and 
recommendations. 


Introduction 

In  recent  years,  the  computer  game  and  simulation  industry  has  grown  fast  and  become  a part  of  American 
culture.  Computer  learning  games  are  not  only  welcomed  as  a way  to  motivate  learners,  but  also  can  serve  as  a 
mechanism  for  learning.  Prensky  (2001,  2006),  Aldrich  (2005),  Gee  (2003,  2005),  and  Shaffer  (2006)  have  written 
extensively  about  the  potential  benefits  of  using  computer  games  for  learning.  According  to  Mitchell  & Savill-Smith 
(2004),  games  assist  all  different  learning  styles  by  supporting  confidence  and  encouraging  multimodal  literacy 
through  the  use  of  video,  audio,  and  text.  Game  players,  especially  children,  are  more  interested  in  games  with  good 
audio  and  video  (Vernadakis  et  ah,  2005).  Kirriemuir  & McFarlane  (2004)  identified  that  the  experience  of  game 
play  seems  to  be  affecting  learners’  expectations  of  learning  activities.  Therefore,  the  idea  of  integrating  computer 
learning  games  into  the  curriculum  is  becoming  an  important  research  topic  in  education. 

The  use  of  computers  has  been  prevalent  in  education  even  at  the  preschool  level.  Much  of  the  research 
conducted  in  preschools  has  looked  at  the  impact  on  the  social  skills  of  preschool  children  (Chen  & Chang,  2006). 
Using  computers  in  a preschool  setting  has  been  found  to  increase  the  motivation  of  young  children  (Mioduser,  Tur- 
Kaspa,  & Leitner,  2000).  However,  there  is  a lack  of  understanding  whether  game-integrated  technology  tools 
indeed  can  enhance  student-learning  outcomes  in  the  area  of  literacy  as  compared  to  technology  tools  without  games 
and  simulations.  There  is  a growing  need  to  understand,  empirically,  how  well  the  learning  with  educational 
computer  games  occur  particularly  in  the  preschool  setting.  Limited  research  has  been  completed  on  the  impact  of 
computer  learning  games  in  preschools  (Chen  & Chang,  2006;  Clements  & Sarama,  2003).  As  a result,  a key  goal  of 
this  study  was  to  investigate  how  computer  games  may  affect  student  learning  outcomes  in  the  preschool  setting. 

Literature  Review 


Computer  Literacy  Games  and  Preschoolers 

In  early  childhood  education,  the  use  of  computer  software  as  both  a successful  learning  tool  and  teaching 
method  has  increased  rapidly  over  the  last  thirty  years  (Blok  et  al.,  2002).  Preschool  children  should  not  be  kept 
away  from  computer  learning  games  based  on  their  age  or  developmental  levels.  Instead  computer  learning  games 
combined  with  traditional  teaching  methods  have  the  potential  to  enhance  children’s  learning  and  help  them  meet 
educational  goals  (Vernadakis  et  ah,  2005). 

Children’s  learning  through  the  use  of  computer  software  is  becoming  viewed  as  a positive  learning  tool 
that  can  be  used  as  a major  contributor  to  a child’s  education  (Plowman  & Stephen,  2007).  Computers  are  becoming 
more  accepted  by  teachers  and  children  in  early  childhood  classrooms  (Hinitz,  1989).  While  there  were  previous 
concerns  about  the  ability  of  very  young  children  to  use  computers,  research  has  shown  that  young  children  can 
become  confident  and  enthusiastic  computer  users  (Plowman  & Stephen,  2007). 

Lonigan  et  ah  (2003)  examined  the  effects  of  Computer  Assisted  Instruction  (CAI)  on  the  phonological 
skills  of  pre-school  age  children  who  were  considered  at-risk  in  their  reading  skills.  Children  who  participated  in  the 
CAI  group  were  found  to  have  stronger  vocabulary  skills,  rhyming  skills,  and  increased  overall  reading  skills  after 
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playing  with  the  computer  game  as  compared  to  the  comparison  group.  Additionally,  the  children  who  participated 
in  the  CAI  rated  the  intervention  program  more  favorably  than  students  who  were  in  the  comparison  group. 
Macarcuso  et  al.  (2006)  studied  the  effect  of  Lexia  Phonics  Based  Reading  Program  (2001)  and  Strategies  for  Older 
Students  (2001)  in  a public  school.  While  Macarcuso  et  al.  did  not  find  a significant  difference  between  the 
experimental  and  control  group,  there  was  a trend  favoring  the  experimental  group.  Macarcuso  et  al.  stated  they  did 
not  find  a significant  difference  due  to  the  large  standard  deviation  within  the  groups.  Macarcuso  et  al.  conducted  a 
secondary  analysis  on  students  who  were  Title  I eligible.  Title  I students  in  the  experimental  group  showed  literacy 
growth  that  Title  I students  in  the  control  group  did  not  make. 

Computer  learning  games  have  been  effective  at  increasing  the  literacy  skills  of  young  children  (Lonigan  et 
al.,  2003;  Macarcuso  et  al.,  2006).  This  research  was  designed  to  determine  if  computer  learning  games  that  are 
available  for  free  through  the  internet  can  increase  the  literacy  skills  of  very  young  children.  Additionally,  this  study 
looked  at  the  impact  of  computer  learning  games  on  the  letter  naming  fluency  and  initial  sound  fluency  skills.  These 
literacy  skills  are  vital  to  students’  success  in  the  area  of  reading  (Kaminski  & Good,  1996,  1998;  Laimon,  1994, 
Martson  & Magnusson,  1988). 

Computer  Game  Features  For  Preschool  Children 

Most  educational  games  are  designed  to  engage  children  in  a virtual  environment  and  through  practice  with 
interactive  elements  children  gain  knowledge  in  the  subject  matter  presented  in  the  game.  The  interactive  exercises 
in  computer  games  should  be  designed  in  such  a way  as  to  be  significantly  different  than  through  traditional 
instruction  (Gredler,  1996).  These  differences  are  often  achieved  with  the  features  of  computer  software  to  engage 
children  and  enhance  their  learning  through  the  use  of  color,  sound,  animation,  audio  and  visual  feedback,  and  age 
appropriateness  (Chute  & Miksad,  1997;  Kirchner,  2002). 

Lonigan  et  al.  (2003)  studied  the  effects  of  Computer  Assisted  Instruction  (CAI)  on  phonological  skills  of 
pre-school  age  children  who  were  considered  at-risk  in  their  reading  skills.  The  study  found,  among  the  software’s 
features  were  highly  digitized  speech  and  colorful  graphic  images  that  kept  the  children’s  attention  while  they 
completed  a series  of  interactive  tasks  within  the  context  of  the  game’s  adventure  setting.  Throughout  the  game 
children  heard  auditory  content  using  headphones.  The  game  provided  clear  instructions  and  tutorials  for  the 
children  to  follow.  This  required  an  active  response  from  the  child.  The  children  were  able  to  review  the  instructions 
at  any  time  during  game  play.  The  game  allowed  the  children  to  practice  seven  phonological  tasks,  which  included 
recognizing  when  words  rhymed  and  matching  words  by  their  first,  middle,  or  last  sounds.  These  features  performed 
an  important  aspect  in  the  children’s’  learning. 

Additionally,  computers  can  assist  learning  by  using  the  computer’s  speed  to  help  children  recognize  words 
when  they  are  quickly  flashed  on  the  screen.  This  helps  the  children  to  see  the  shapes  of  the  letters  and  how  they  are 
grouped  together  to  form  words.  A study  was  conducted  to  measure  the  accuracy  rate  of  word  recognition  by 
children  who  saw  a word  flashed  on  the  screen  for  a limited  period  of  time  (Blok  et  al.,  2002).  A sixty-seven  percent 
accuracy  rate  was  achieved  consistently  with  all  the  participants  who  used  the  computer  program.  While  similar  to 
the  traditional  educational  method  of  using  flash  cards,  the  use  of  a computer  for  this  method  of  instruction  allowed 
for  a consistent  interval  of  time  for  each  word  (Blok  et  al.,  2002). 

Audio  feedback  for  early  childhood  educational  software  is  important  to  provide  encouragement, 
reinforcement  and  modeling  when  needed  (Chute  & Miksad,  1997).  Nikolopoulou  (2007)  noted  that  feedback  to 
right  and  wrong  responses  should  be  provided  in  a pedagogically  appropriate  way  so  children  are  encouraged  to 
continue  on  their  work.  Many  times,  audio  feedback  as  a feature  of  the  computer  program  can  provide  feedback  at  a 
rate  that  a teacher  may  not  be  able. 

Children  who  have  been  raised  in  the  culture  of  video  games  and  computers  compared  to  past  generations 
require  more  stimuli  to  keep  them  involved  in  the  learning  process  (Gredler,  1996).  Keeping  this  in  mind,  making 
early  childhood  software  can  be  a difficult  task,  but  knowing  what  features  to  implement  and  what  features  to  look 
for  in  software  is  a significant  step.  When  all  the  features  of  color,  sound,  animation,  gender  preference,  audio  and 
visual  feedback,  and  age  appropriate  are  used  properly,  well-designed  software  is  made.  This  type  of  software 
allows  children  to  learn  through  active  exploration  and  interaction  (Lonigan  et  al.,  2003).  A secondary  purpose  of 
this  study  was  to  determine  the  features  of  the  learning  games  (i.e.,  Starfall  and  AOL  Leam  Your  Letters)  that  had 
an  impact  on  the  engagement  of  the  young  children.  Engagement  in  an  instructional  task  is  vital  for  the  task  to  have 
the  desire  impact  on  learning. 
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Teacher  Perceptions  Of  Computer  Software  Features 


Medvin  et  al.  (2002)  examined  thirty-eight  teachers  and  day  care  providers’  characteristics  influencing  the 
frequency  of  computer  use  in  their  preschool  classrooms.  The  findings  showed  that  the  teachers  who  had  more 
training  were  more  likely  to  have  higher  levels  of  motivation  for  computer  use  in  the  classroom.  The  results  also 
found  that  teachers  who  had  both  quality  and  quantity  prior  experience  with  computers  were  influenced  by  their  self- 
efficacy  and  anxiety.  Teachers  viewed  computers  as  an  important  material  in  the  classroom,  but  still  had  anxious 
feeling  about  using  them.  After  the  workshops  in  the  study,  the  teacher’s  computer  anxiety  reduced  while  self- 
efficacy  and  values  enhanced.  Chen  and  Chang  (2006)  conducted  a study  that  gathered  information  about  297  state 
pre -kindergarten  and  their  attitudes,  skills,  and  instructional  methods  towards  the  use  of  computers.  The  results  of 
the  study  found  that  early  childhood  teachers  are  not  ready  to  integrate  computers  into  their  classrooms  yet  because 
more  than  half  of  the  teachers  felt  unconfident  in  using  computers  in  their  classrooms.  In  addition,  less  than  a third 
of  teachers  demonstrated  the  skills  to  select  appropriate  children’s  educational  software  for  their  classrooms  and 
only  half  of  the  teachers  felt  comfortable  teaching  their  students  how  to  use  computers. 

The  last  purpose  of  this  study  was  to  determine  if  any  of  the  features  of  the  learning  games  had  an  impact 
on  the  perceptions  of  the  teachers  in  the  classroom.  Positive  teacher  perception  of  a computer  learning  game  is 
important  for  the  teacher  to  decide  to  use  classroom  time  for  the  game.  However,  positive  teacher  perception  does 
not  necessarily  equate  to  an  effective  program. 


Research  Questions 

This  study  had  three  purposes  and  from  these,  the  research  questions  were  developed.  The  research 
questions  were: 

1 . Is  there  any  significant  increase  in  literacy  skills  of  the  two  groups? 

2.  What  features  of  the  learning  games  may  enhance  student  engagement? 

3.  What  features  of  the  learning  games  enhanced  teacher  perceptions  of  student  satisfaction? 

Methods 

Participants 

The  study  started  with  21  children  and  ended  up  with  15  children  who  participated  throughout  the  entire 
study.  Six  students  were  lost  due  to  attrition.  This  was  because  the  children  no  longer  received  services  at  the 
preschool.  The  age  ranges  of  the  15  participants  were  34  to  60  months  old  and  they  attended  a Child  Development 
Center  preschool.  The  school  had  a full  time  staff  and  taught  pre-school  level  education  to  children.  The  children 
were  selected  based  on  their  age  and  enrollment  in  the  pre-school.  All  of  the  parents  of  the  children  consented  to 
having  their  children  participate  in  the  study. 

Researchers 

A five -person  research  team  included  two  faculty  advisors,  one  from  the  computer  science/information 
technology  department  and  one  from  the  teacher  education  department,  conducted  this  study.  The  three  student 
researchers  participating  included  one  from  the  liberal  studies  department,  one  from  the  computer  science 
department,  and  one  student  from  the  communication  design  department. 

Intervention 

The  computer  learning  games  used  in  the  study  were  Starfall  and  A OL  Learn  Your  Letters.  Both  games  are 
early  literacy  games  designed  for  preschool  children  and  can  be  accessed  free  over  the  Internet.  These  games  were 
chosen  because  the  content  was  appropriate  for  the  age  of  the  children  in  the  study  and  they  were  easily  accessible 
from  online  sources  at  no  cost. 

Starfall.  The  authors  of  Starfall  report  that  it  teaches  early  letter  recognition  skills  and  literacy  through  the 
use  of  sound,  color,  animation,  and  simple  games.  Children  are  able  to  choose  from  different  letters  in  the  alphabet 
and  a series  of  activities  focuses  on  the  letter  chosen.  Starfall  was  created  in  2002  by  a team  of  educators  and 
designers.  This  program  addresses  the  assessed  skills  of  letter  naming  fluency  and  initial  sound  fluency. 
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America  Online  (AOL)  Learn  Your  Letters.  The  authors  of  AOL  Learn  Your  Letters  report  that  the  game 
teaches  letter  recognition  skills  by  asking  children  to  choose  the  words  that  start  with  a letter  shown  to  them  at  the 
bottom  of  the  screen.  This  program  addresses  the  assessed  skills  of  letter  naming  fluency  and  initial  sound  fluency. 

After  a review  of  the  literature  and  an  analysis  of  each  game,  the  researchers  identified  features  in  each  of 
the  games  to  be  evaluated.  These  were  evaluated  on  a scale  of  1 to  5,  with  1 representing  an  almost  non-existent 
feature  and  5 representing  a very  prominent  feature  in  the  game  (See  Table  1).  Features  that  were  shown  to  not  be 
significantly  different  in  either  game  were  removed  from  the  table.  This  significance  was  determined  by  comparing 
the  average  ranking  of  the  AOL  game  features  from  the  average  ranking  of  the  Starfall  features.  As  seen  in  the  table, 
all  identified  features  were  found  to  be  more  predominant  in  the  Starfall  game. 

Table  1.  Differences  in  Features 


Feature 

Starfall  ? 

lOL  I 

)iffe  rence 

Randomness  of  game  play  (different  scenarios 
depending  on  student  choices) 

3.4  1 

.4 

2 

Engagement  4. 

8 

2.2 

2.6 

Student  can  explore  game  on  their  own  at  their 
own  pace  (non-linear  game  play) 

4 ( 

I .6 

3.4 

Real  world  activities  and  situations  drawn  from 
life 

4.2  ( 

i.  8 

3.4 

Surprise  elements 

4.4 

2.4 

2 

Challenging  4 

2 

2 

Procedure 

The  parents  of  the  children  at  the  Child  Development  Center  signed  consent  forms  for  their  children  to 
participate  in  the  research  study.  All  children  were  pretested  prior  to  the  study  beginning.  The  tests  used  for  the  pre- 
test were  the  two  subtests  from  the  Dynamic  Indicators  of  Basic  Early  Literacy  Skills  (DIBELS).  The  pretest 
consisted  of  an  initial  sound  fluency  component  and  a letter  naming  fluency  component.  Each  child  was  tested  by 
one  of  the  five  researchers.  The  guidelines  and  procedures  of  the  tests  were  followed. 

Two  computers  were  set  up  at  the  center  and  the  children  were  randomly  assigned  into  two  groups.  The 
first  group  completed  the  intervention  using  Starfall.  The  second  group  completed  the  intervention  using  AOL  Learn 
Your  Letters.  All  children  were  exposed  to  the  early  literacy  instruction  of  the  preschool.  The  games  were  installed 
on  the  computers  in  a way  so  that  the  children  only  had  access  to  the  games  and  not  to  the  Internet.  The  names  of  the 
children  were  posted  on  the  desk  next  to  each  computer  so  they  and  staff  members  would  have  a visual  reference  of 
which  children  were  on  each  computer.  A sign  up  sheet  was  created  for  each  child’s  name  to  be  checked  off  as  they 
played  each  day.  Children  were  assigned  to  play  their  game  for  ten  minutes.  After  ten  minutes,  the  next  child  played 
the  game  until  all  of  the  children  had  played  each  day.  The  study  ran  for  six  weeks,  five  days  a week.  Children  who 
refused  to  engage  in  the  computer  game  were  not  forced  to  play. 

The  researchers  ran  the  first  two  weeks  of  the  intervention.  This  served  as  a model  to  the  CDC  staff  as  to 
how  the  intervention  should  be  implemented.  The  staff  of  the  CDC  center  conducted  the  rest  of  the  observation  with 
a student  researcher  making  fidelity  checks  twice  a week.  The  CDC  staff  recorded  on  the  sign-in  sheet  whether  the 
children  played  the  game  or  did  not  play  due  to  absence  or  other  reasons. 

At  the  conclusion  of  the  study,  the  children  were  given  post-tests.  An  alternative  form  of  the  DIBELS 
assessment  was  used  for  the  post-test.  Each  child  was  tested  on  initial  sound  fluency  and  letter  naming  fluency. 
Additionally,  the  pre-school  staff  and  researchers  filled  out  surveys  for  each  game. 

Results 

Ql.  Is  there  any  significant  increase  in  literacy  skills  of  the  two  groups? 

Through  the  data  analysis  and  T-test  results,  a significant  difference  in  literacy  skills  between  the  computer 
games  Starfall  and  AOL’s  Learn  Your  Letters  was  found.  A total  of  six  T-tests  were  run  with  the  data.  For  each 
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computer  game,  the  pre-test  and  post-test  scores  were  compared  separately,  for  both  initial  sound  fluency  and  letter 
naming  fluency. 

In  the  comparison  of  the  Letter  Naming  Fluency  pre-test  and  post-test  scores,  a significant  difference  was 
found.  When  compared  all  15  kids’  pre-test  scores  to  their  post-test  scores,  the  one-tailed  paired  sample  t-test 
returned  a p-value  of  0.016,  which  is  statistically  significant  at  the  conventional  significance  level  of /?<0.05.  This 
result  indicated  that  both  learning  games  increased  the  children’s  letter  naming  fluency  skills  in  general.  To  further 
understand  the  learning  outcomes  at  each  game  level,  the  researchers  compared  the  pre-test  scores  with  the  post-test 
scores  for  each  game  group.  Of  the  t-test  of  the  preschoolers  who  played  the  AOL’s  Learn  Your  Letters  game 
showed  a /?- value  of  0.023  (p<0.05)  while  the  Starfall  group  returned  a /?- value  of  0.078.  These  results  indicated  that 
the  AOL  game  was  more  effective  in  increasing  children’s  letter  naming  fluency  than  the  Starfall  game. 

When  comparing  the  growth  of  initial  sound  fluency  there  was  no  significant  growth  difference  between 
the  two  groups.  Also,  when  comparing  the  growth  of  letter  naming  fluency  for  the  computer  games  there  was  no 
significant  growth  difference. 

Overall,  the  results  showed  a significant  difference  in  literacy  skills  of  the  letter  naming  fluency  test  for  the 
computer  game  AOL  Learn  Your  Letters.  The  other  computer  game,  Starfall,  showed  no  statistically  significant 
results  in  either  of  the  letter  naming  fluency  test  nor  the  initial  sound  fluency  test. 

Q2.  What  features  of  the  learning  games  may  enhance  student  engagement? 

The  Child  Development  Center  staff  and  the  researchers  who  observed  the  study  responded  to  a satisfaction 
survey.  Questions  were  asked  regarding  student  engagement  for  each  game.  The  responses  were  ranked  on  a 1 to  5 
scale,  with  1 representing  low  student  engagement  observed  and  5 representing  high  student  engagement  observed. 
Table  2 shows  the  average  score  for  each  engagement  question  asked  for  each  game. 

Table  2.  Teacher  Rating  of  Student  Engagement  and  Satisfaction 


Engagement  and  Satisfaction  Questions 

Starfall 

AOL 

The  learning  game  is  visually  appealing 

5 

2.8 

The  audio  is  appropriate 

4.8 

3.2 

The  animation  is  appealing  for  my  students 

5 

2.8 

It  plays  smoothly  during  use 

4.7 

4.5 

The  interactive  features  are  appropriate  for  my  students 

4.3 

3 

Overall,  it  is  easy  to  use 

4.3 

2.7 

Overall,  my  students  are  engaged  when  playing  the  game 

4.7 

1.3 

Overall,  my  students  are  happy  to  play  the  game 

4.7 

1.2 

The  above  results  show  a significant  difference  in  the  perceived  engagement  of  the  children  between  the 
two  games.  Starfall  was  observed  to  engage  the  children  and  keep  their  attention  more  than  AOL  Learn  Your 
Letters.  Starfall  was  seen  to  be  significantly  more  visually  appealing,  had  better  audio,  better  interactive  features, 
and  was  easier  to  use.  All  of  these  features  contributed  to  the  children  being  more  engaged  with  the  Starfall  game. 
The  survey  respondents  also  observed  that  the  children’s  enjoyment  of  the  Starfall  game  was  much  higher  than  the 
AOL  game. 

Q3.  What  features  of  the  learning  games  enhanced  teacher  perceptions  of  student  satisfaction? 

Six  different  educators  and  assistants  at  the  pre-school  were  asked  to  fill  out  a questionnaire  regarding  the 
two  computer  learning  games.  Of  the  two  programs,  Starfall  was  the  unanimous  choice  in  terms  of  student 
engagement  and  ease  of  use,  earning  an  average  score  of  4.5  out  of  5.  AOL’s  Learn  Your  Letters  received  a much 
lower  approval  rating,  averaging  a 2.5  out  of  5.  AOL’s  hardest  shortcomings  were  in  the  student  engagement 
category  (See  Table  2).  AOL  Learn  Your  Letters  was  rated  1.2  out  of  5 on  “Student  Happiness”  and  1.3  out  of  5 on 
“Student  Engagement”.  Respondents  indicated  they  did  not  believe  that  the  students  learned  the  content  by  playing 
the  game,  averaging  a 1.7  out  of  5.  Clearly,  the  educators  favored  the  Starfall  game  because  of  its  high  level  of 
student  engagement. 
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Eight  T-tests  were  performed  on  each  of  the  eight  teacher  ratings,  comparing  AOL  Learn  Your  Letters  to 
Starfall.  In  t-test  results,  AOL  Learn  Your  Letters  scored  significantly  lower  than  Starfall  in  the  categories  of  visual 
appeal  (p-value  0.015),  animation  appeal  (p-value  0.003),  ease  of  use  (p-value  .04),  student  engagement  (p-value 
0.00002)  and  student  happiness  ( p-value  0.00001).  Starfall  was  rated  to  have  superior  visual  appeal  due  to  its  rich 
and  colorful  visual  style.  Starfall’s  color  palette  is  bright  and  features  primary  colors.  The  AOL  game  features  a dull 
purple-grey  color  scheme  that  may  not  have  appealed  to  the  children.  Starfall  scored  higher  on  audio  as  well, 
because  the  game  featured  several  dozen  audio  clips  of  sound  effects  and  human  voices.  Each  animation  was 
accompanied  by  an  appropriate  sound  effect.  AOL  used  repetitive  sound  clips  and  a short  music  loop,  which  may 
have  caused  the  children  to  become  bored.  Starfall  also  used  superior  animations.  There  were  two  to  five  animated 
sequences  for  each  letter  in  the  alphabet,  and  the  children  would  often  repeat  the  same  letter  to  see  the  animations 
again.  AOL  had  two  main  animations:  a transition  animation,  and  one  looped  animation  of  a flying  cow.  After  have 
the  children  watch  these  same  two  animations  over  and  over,  they  appeared  to  grow  bored  and  wanted  to  stop 
playing. 

Starfall  was  rated  slightly  higher  than  AOL  on  its  interactive  features  because  Starfall  offered  a wide 
variety  of  mini  games  for  the  students  to  play.  This  interactivity  appeared  to  engage  the  students  more  than  AOL’s 
limited  interactivity,  which  required  the  students  to  click  matching  letters  and  then  play  a repetitive  item-collection 
game.  The  pre-school  teachers  rated  Starfall  higher  on  ease  of  use,  although  neither  group  of  children  appeared  to 
have  any  confusion  or  difficulty  in  playing.  AOL  may  have  received  a low  score  in  this  category  due  to  the  fact  that 
it  scored  low  in  all  the  other  categories.  AOL’s  other  poor  scores  may  have  influenced  this  score  as  well.  AOL’s  two 
lowest  marks  were  in  student  satisfaction  and  happiness.  This  is  clear,  as  several  students  refused  to  play  the  game 
because  it  was  not  engaging  enough.  Students  playing  the  Starfall  game  appeared  to  be  happy  and  deeply  engaged. 
Many  of  the  students  expressed  complaints  when  their  Starfall  turn  was  over,  but  the  AOL  children  seemed  to  be 
relieved  or  indifferent  when  asked  to  end  their  turn. 


Discussion 

The  data  collected  showed  a growth  in  the  preschooler’s  letter  naming  skills  that  played  the  AOL  Learn 
Y our  Letter  computer  game  only.  These  results  were  unexpected  due  to  the  engagement  observations  and  the 
efficacy  questionnaire  that  compared  the  two  computer  games.  Starfall  was  rated  higher  than  AOL’s  Learn  Your 
Letters  in  both  engagement  and  efficacy.  The  computer  game  (i.e.,  AOL  Learn  Your  Letters)  that  showed  significant 
enhancement  of  letter  naming  skills  was  the  computer  game  ranked  low  in  engagement  and  efficacy.  This  is  an 
valuable  finding  as  it  is  important  to  realize  that  learning  games  that  have  high  levels  of  engagement  and  features 
(e.g.,  colors,  sounds,  animation)  may  not  be  the  best  choice  educationally. 

Starfall  was  rated  as  more  engaging  for  the  children  to  play  than  the  AOL  Learn  Your  Letters  game.  This  is 
significant  due  to  the  fact  that  the  children  who  played  the  less  engaging  game  did  show  significant  improvement  in 
their  letter  naming  fluency.  Engagement  has  been  seen  as  an  important  factor  in  the  ability  of  a computer  game  to 
educate;  however,  in  this  study,  the  Starfall  game  was  observed  to  be  significantly  more  engaging,  yet  it  did  not 
produce  a significant  educational  benefit.  This  is  likely  due  to  the  fact  that,  while  AOL  Learn  Your  Letters  was 
observed  to  be  less  engaging,  it  did  provide  repeated  exposure  to  multiple  letters  during  the  time  allotted  for  each 
child  to  play  the  game.  Starfall  while  possibly  more  entertaining  took  much  longer  per  letter  for  the  child  to  progress 
through  the  game.  In  a ten-minute  session,  a child  might  be  exposed  to  one  or  two  letters  on  Starfall  while  the  child 
on  the  AOL  Learn  Your  Letters  game  may  have  been  exposed  to  four  letters  or  more.  AOL  Learn  Your  Letters  also 
randomly  generated  the  letters,  while  the  children  could  choose  a letter  while  playing  Starfall.  This  could  lead  to  the 
possibility  that  the  children  playing  Starfall  chose  to  view  the  same  letter  or  letters  each  session,  while  the  children 
playing  AOL  Learn  Your  Letters  were  exposed  to  random  letter  the  program  generated.  This  may  have  increased  the 
number  of  letters  to  which  the  children  in  the  AOL  Leam  Your  Letters  group  were  exposed  to. 

The  staff  of  the  Child  Development  Center  reported  to  the  researchers  and  documented  on  the  sign-in 
sheets  as  well  as  observed  during  fidelity  checks,  that  the  students  in  the  second  group  more  often  refused  to  play 
AOL  Learn  Your  Letters  because  they  did  not  like  or  want  to  play  it.  This  phenomenon  was  not  seen  in  the  other 
group  which  played  Starfall.  The  researchers  also  observed  that  the  children  who  did  play  the  game  seemed  to  be 
easily  distracted  while  playing.  The  children  who  played  the  AOL  game  had  post-test  scores  in  letter  naming  fluency 
that  were  higher  than  their  pre-test.  Despite  a perceived  lack  of  engagement  with  AOL  Learn  Your  Letters,  an 
educational  benefit  was  still  found. 

The  high  level  of  engagement  in  the  Starfall  game  was  detected.  The  pre-school  children  openly  showed 
their  feelings  about  the  games.  The  children  had  no  problem  with  telling  their  teacher  that  they  did  not  want  to  play 
AOL  Learning  Letters.  In  comparison,  the  Starfall  group  of  children  had  very  high  attendance  and  almost  always 
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eagerly  sat  down  to  play  the  game.  AOL  Learn  Your  Letters  earned  very  low  marks  for  “Learned  Content”  most 
likely  due  to  the  low  engagement  level  of  the  students.  This  assumption  ignores  the  quality  of  the  material  learned 
during  the  very  short  periods  of  engagement  on  the  AOL  game.  Although  the  students  were  far  less  engaged,  they 
appear  to  have  absorbed  higher  educational  content. 

The  findings  of  this  study  seemed  to  confirm  the  meta-analysis  results  of  Means  et  al.  (2009)  in  one  aspect, 
“. . . the  expansion  of  time  on  task  for  online  learners  were  the  only  statistically  significant  influences  on 
effectiveness.”  The  Starfall  game  was  certainly  engaged  the  preschool  children,  but  the  games  such  as  the  matching, 
drop  and  drag,  watching  the  animations  often  took  a very  long  time  to  complete  and  the  time  consuming  parts  were 
often  off  task  activities.  For  example,  when  you  click  on  the  letter  “C”  in  the  Starfall  game,  there  is  a cat  that  will 
climb  up  and  down  the  edges  of  the  rectangle  display  area,  which  takes  up  precious  educational  time.  It  is  clearly  an 
off  task  engagement  since  it  shows  a cat  running  with  an  irrelevant  background  music.  On  the  other  hand,  the  AOL 
Learn  Your  Letters  game  forced  the  children  to  be  on  task  by  minimizing  off  task  activities.  Based  on  this 
observation,  we  conclude  that  on  task  engagement  is  critical  for  student  learning  outcomes.  Off  task  engagement 
does  not  seem  to  have  a direct  impact  on  the  learning  outcomes,  although  it  may  help  increase  student  satisfaction 
and  overall  interest  toward  the  subject  matter.  Of  course,  it  is  important  to  point  out  that  enhancing  study  interest 
and  increasing  their  positive  attitude  toward  learning  are  important  aspects  of  education.  Some  may  even  say  that 
increasing  satisfaction  and  nurturing  interest  are  more  important  than  the  actual  subject  matter  learning  for  young 
kids.  Flopefully,  our  findings  can  help  increase  educational  game  designers’  awareness  on  the  importance  of 
integrating  more  on  task  engagement  activities  for  effective  learning  outcomes. 

Limitations  and  Suggestions  for  Further  Research 

Based  on  the  results  of  this  study  the  following  areas  are  suggested  for  further  research.  Further  research 
should  focus  on  variations  of  this  study  including  longer  intervention  and  maintenance  periods,  a larger  sample  size, 
and  participants  of  different  ages.  In  this  manner,  research  can  ascertain  the  impact  of  a technology-based 
intervention  overtime,  on  a larger  population,  and  at  varying  age  levels. 
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Abstract 

On-line  barriers  often  limit  the  accessibility  WebPages  for  students  with  disabilities.  The  purpose  of  this 
study  was  to  evaluate  the  accessibility  of  Georgia  districts’  home  pages.  The  results  of  the  study  show  that  from  a 
list  of  244  district  websites,  none  of  them  were  accessible.  Along  with  the  results,  including  specific  reoccurring 
problems,  the  presentation  will  provide  the  audience  with  information  about  how  to  correct  typical  accessibility 
problems. 


Messages  in  a Bottle:  Accessibility  of  Georgia  Districts’  Website  Homepages 

The  Americans  with  Disabilities  Act  (ADA)  of  1990  provides  the  same  civil  rights  protection  to  individuals 
with  disabilities  individuals  with  disabilities  as  other  federal  laws  that  prevent  discrimination  on  the  basis  of  race, 
gender,  national  origin,  and  religion  (Button  & Wobschall,  1994).  Title  III  of  the  ADA  directs  that  public  facilities 
make  reasonable  modifications  to  control  discrimination  and  support  accessibility  in  policies,  practices,  and 
procedures  (Council  for  Exceptional  Children,  1994).  As  a result  of  this  landmark  legislation,  accessibility 
alterations  such  as  providing  ramps  to  elevated  areas  and  providing  accessible  signage  through  height  adjustments 
and  raised  lettering  have  become  commonplace  across  the  United  States. 

The  potential  of  technology  to  impact  student  learning  is  consistent  with  ideas  about  developmentally 
appropriate  teaching  and  learning.  Further,  access  to  technologies  addresses  important  learning  targets  that  focus  on 
higher  order  thinking  skills  such  as  drawing  analogies,  inferences,  and  cognitive  mapping  (Quinn  & Valentine, 

2002).  In  writing  the  introduction  for  the  Association  for  Supervision  and  Curriculum  Development’s  yearbook, 
Chris  Dede  (1998)  advocates  that  computer  tools,  which  include  access  to  the  Internet,  are  a powerful  means  to 
improve  educational  outcomes  for  all  students.  He  further  argues  that  technology  can  provide  success  for  all  students 
through  special  measures  to  aid  people  with  disabilities  and  those  referred  to  as  “disenfranchised.” 

The  World  Wide  Web  (WWW)  has  become  an  invaluable  resource  for  many  people  with  disabilities. 
Accessibility  across  platforms  and  geographic  distance  makes  the  WWW  an  ideal  universal  tool  for  gathering  and 
disseminating  information  (Heflich  & Edyburn,  1998).  Today,  the  Word  Wide  Web  is  almost  universally  used  to 
disseminate  information  about  a school  or  a school  district.  Wong  (1997)  discussed  using  the  Internet  for  increased 
self-advocacy  by  individuals  with  physical  impairments.  It  is  ironic,  however,  that  while  technological 
developments  have  enhanced  and  provided  new  exciting  opportunities  for  the  WWW,  they  have,  simultaneously, 
complicated  and  limited  the  accessibility  of  the  content  and  resources  for  individuals  with  disabilities. 

Physical  barriers  are  obvious  accessibility  concerns.  Web  page  developers  need  to  be  aware  that  on-line 
barriers  can  create  significant  problems  for  some  users.  The  Americans  with  Disabilities  Act  requires  that  all 
organizations  make  reasonable  accommodations  for  individuals  with  disabilities.  With  recent  judicial  rulings  such  as 
those  against  Target  Corp  (Selvin,  2007)  home  page  developers  should  work  towards  designing  and  building  Web 
sites  that  are  accessible  to  all  individuals.  It  is  important  that  Web  page  developers  use  and  follow  standards  that 
allow  accessibility  to  all  WWW  users. 

Method 

To  examine  the  accessibility  of  Georgia’s  district  website  home  pages  a descriptive  study  was  conducted. 
An  online  school  directory  of  school  websites  from  Georgia  provided  URLs  for  the  244  website  homepages 
analyzed  for  this  study.  Bobby  4.0.1  (Center  for  Applied  Special  Technology,  2000),  and  the  ATRC  Web 
Accessibility  Checker  (Adaptive  Technology  Resource  Centre,  2009)  were  used  to  analyze  each  website  homepage. 
Both  tools  allow  researchers  and  other  professionals  to  evaluate  Web  pages  in  accordance  with  the  W3C  Web 
Accessibility  Initiative's  guidelines. 

The  results  of  this  analysis  show  that  100%  of  the  home  pages  analyzed  by  Bobby  4.0.1  and  99%  of  the 
pages  analyzed  by  ATRC  were  not  accessible.  This  means  that  at  least  one  Priority  1 error  (seriously  affects 
accessibility)  was  detected  on  these  pages. 

Discussion 

Web  developers  who  are  developing  WebPages  for  school  districts  need  to  examine  their  Web  sites  for 
accessibility  problems.  It  is  strongly  recommended  that  validation  methods  be  used  in  the  early  stages  of  Web 
development,  which  will  help  make  problems  easier  to  correct  and  assist  developers  in  avoiding  many  accessibility 
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problems.  In  addition  to  evaluation  tools  such  as  Bobby  and  ATRC  expert  and  novice  users  with  disabilities  should 
be  invited  to  view  home  pages  and  provide  feedback  about  accessibility  or  usability  problems  and  their  severity. 
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Elements  of  Successful  Internships 
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Introduction. 

Advanced  programs  in  instructional  technology  are  charged  with  the  responsibility  of  preparing 
instructional  designers,  professional  development  specialists,  teachers,  and  technology  facilitators  as  they  enter 
positions  in  business,  industry,  the  military,  and  K12  education.  The  internship  for  professional  programs  is  a broad 
topic  that  covers  many  disciplines,  yet  there  are  common  elements  across  all  professions.  One  of  the  strongest 
attributes  of  a successful  internship  experience  is  the  application  of  theoretical  constructs  learned  in  academic 
settings  to  best  practices  in  future  work  environments  (Doucherty,  2009;  Stichman  & Farkas,  2005).  The  intern  must 
also  articulate  how  their  knowledge  base  has  evolved  into  practical  application  in  their  profession  (Cunningham  & 
Sherman,  2008).  Successful  internships  are  supported  through  strong  mentoring,  both  on-site  and  by  academic 
supervisors  (Conn,  Roberts,  & Powell , 2009).  Other  characteristics  that  result  in  good  outcomes  include  carefully 
designed  seminar  courses  with  a theory-based  approach  (Hsu,  2004);  documentation  of  internship  hours  and 
activities  (Macchia  & Freedman,  2004);  a criterion  based  final  report  reviewed  by  faculty  (Cunningham  & Sherman, 
2008);  and  alignment  with  professional  standards  (Early  & Glenn,  2009).  Accreditation  standards  have  an  important 
function  to  ensure  interns  meet  basic  knowledge  and  skills  associated  with  their  profession.  State  and  national 
standards  are  used  to  assess  competency  and  readiness  for  the  workplace  (Darling-Hammond,  2007).  Institutions 
affiliated  with  the  National  Council  for  the  Accreditation  of  Teacher  Education  (NCATE)  adhere  to  standards 
recommended  by  a Specialized  Professional  Association  (SPA)  such  as  AECT  and  1STE.  These  Specialized 
Professional  Associations  assess  content  knowledge,  abilities  in  professional  activity,  and  evaluation  of  clinical 
practice  (NCATE,  online).  Evaluation  of  clinical  (internship)  experiences  is  often  accomplished  through  use  of 
artifacts  supported  by  student  reflective  writings  (Ring  & Foti,  2003). 

Purpose  for  the  Study. 

At  our  university,  internships  are  designed  as  a culminating  experience  based  on  earlier  course  assignments 
that  are  continuously  applied  to  K12  classrooms.  The  design  of  our  courses  is  supported  by  empirical  research,  thus 
professional  internships  are  planned  using  research-based  ideologies.  Included  in  the  internship  experiences  are 
guided  discussions  that  help  the  intern  make  a personal  connection  between  activities  in  the  schools  with  theoretical 
constructs  under-girding  the  design  of  the  internship.  In  addition  to  discussion  threads,  students  prepare  reflective 
writings  that  focus  on  requirements  for  advanced  licensure  as  a technology  facilitator.  Because  the  master’s  degree 
meets  state  accreditation  requirements,  graduates  earn  an  approved  licensure  area  for  elementary  and  secondary 
schools.  Evidence  to  show  competence  in  areas  of  licensure  appears  in  artifacts  with  reflections  archived  in  an 
electronic  portfolio.  The  portfolio’s  reflective  writings  articulate  alignment  of  standards  and  activities  appropriate 
for  the  K12  learning  environments,  and  reflect  commonly  held  beliefs  in  the  profession  as  defined  in  the  published 
literature.  Thus,  interns  should  have  a clear  and  consistent  understanding  in  how  the  research  supports  best  practice 
in  the  field.  The  purpose  for  this  investigation  is  to  evaluate  how  well  students  in  an  online  seminar  are  able  to 
demonstrate  a strong  connection  between  standards  : research  base  : internship  experiences.  In  addition  to  theory  to 
practice  experiences,  the  investigation  was  designed  to  evaluate  onsite  experiences  related  to  the  work  of  technology 
facilitators  in  K12  schools.  Functions  for  this  position  vary  by  state  and  region;  however,  based  on  national 
standards  for  Technology  Facilitators  most  schools  systems  expect  expertise  in  three  to  four  job  functions.  These 
include:  1.  planning  staff  development  for  building-level  teachers,  2.  modeling  appropriate  use  of  technology  in 
classroom  instruction,  and  3.  leading  initiatives  related  to  technology  programs  in  a school  system  (Brooks-Young, 
2002;  Hofer,  Chamberlin,  & Scot,  2004).  Thus  an  important  part  of  this  investigation  was  to  ensure  interns  have 
opportunities  for  a comprehensive  experience  needed  in  their  future  jobs. 

Methods. 

Design-based  research  is  useful  for  matching  what  the  literature  reports  as  best  practices  within  a variety  of 
settings  and  contexts  with  what  is  actually  occurring  within  a given  educational  setting  (Reeves,  Herrington,  & 
Oliver,  2005;  Sandoval  & Bell,  2004;  Wang  & Hannafin,  2005).  Based  on  reports  in  the  literature,  internship 
experiences  are  grounded  in  theory  yet  authentic  and  interactive.  Interns,  in  any  profession,  use  what  has  been 


41 


learned  in  their  coursework  to  solve  real-world  problems.  This  researcher  reviewed  21  articles  with  empirical 
research  as  well  as  practitioner  journals  reporting  anecdotal  experiences  with  professional  internships.  Keywords 
were  used  to  search  databases  from  a variety  of  disciplines — internship,  experiential  learning,  staff  development, 
professional  development,  technology  facilitators,  design-based  research,  grounded  theory,  and  community  of 
learners.  Based  on  a review  of  the  literature,  two  main  themes.  Theory  to  Practice  and  Community  of  Learners, 
emerged  providing  a framework  for  the  design  of  this  study. 

Theme  1 : Theory  to  Practice. 

One  of  the  dominate  characteristics  of  the  successful  internship  is  the  strong  emphasis  on  interns  being  able  to 
apply  theoretical  constructs  from  classrooms  and  coursework  to  best  practices  in  the  work  environment  (Kennedy, 
1987;  Stichman  & Farkas,  2005).  Not  only  must  the  intern  make  connections  between  theory  and  practice,  he  or  she 
must  also  articulate  how  their  knowledge  base  has  evolved  into  practical  application  in  their  profession 
(Cunningham  & Sherman,  2008).  Ackerman  and  others  (2009)  report  a study  in  which  medical  interns  gain 
professional  identity  while  reflecting  on  critical  events  with  patients.  The  connection  between  information  learned 
through  textbooks  to  critical  incidences  experienced  during  the  internship  lead  to  deep  understanding  of  their 
professional  role  in  healing  the  sick.  Medical  interns  in  Ackerman's  study  met  in  focus  groups  to  reflect  and  discuss 
experiences.  This  type  of  dialog  might  also  be  replicated  in  an  online  environment. 

Macchia  and  Freedman  (2004)  reported  internship  experiences  for  school  library  media  specialists  assigned  to  a 
variety  of  locations.  Interns  communicated  with  the  university  supervisor  using  email  and  discussion  threads.  The 
conclusions  from  this  study  further  support  design  of  course  work  and  instruction  “relevant  to  the  professional 
workplace”  (p.  208).  For  the  school  library  media  specialist,  settings  will  vary.  Each  school  site  provides  unique 
circumstances  that  require  mentoring  as  interns  interpret  what  has  been  learned  in  library  school  with  problems 
encountered  in  their  internship. 

Conn,  Roberts,  and  Powell  (2009)  suggest  an  effective  method  for  mentoring  interns  is  through  an  established 
learning  community.  Peer-interns  may  be  assigned  to  a site  in  a different  county  or  region  of  the  state,  but  come 
together  as  a Community  of  Practice  through  mutual  support  and  knowledge  sharing.  For  students  in  remote  areas  or 
enrolled  in  distance  education  programs,  the  use  of  email,  teleconferencing,  and  discussion  forums  has  become  the 
lifeline  between  the  university  supervisor  and  the  intern  in  the  field.  In  traditional  graduate  courses,  interns  are  “out 
in  the  field”.  University  faculty  might  visit  schools  and  internship  sites  but  also  must  rely  heavily  on  reports  coming 
from  onsite  mentors  and  the  documentation  provided  by  the  student-intern.  Internships  designed  within  an  online 
environment,  using  digital  communication  tools,  might  provide  a better  learning  experience  for  mentoring  and 
feedback. 

Theme  2:  Community  of  Practice  (CoP). 

Education  literature  reports  many  successful  venues  for  learning  communities.  Teachers,  undergraduate 
pre-service  interns,  and  graduate  interns  have  experienced  supportive  environments  in  which  participants  hold 
each  other  accountable,  encourage  the  struggling  learner,  and  acknowledge  those  who  excel  in  achievements 
(Hough,  2004;  Varrati,  2009).  Within  the  CoP,  interns  can  experience  positive  interdependence  among  group 
members.  A common  way  of  life  (rather  than  simply  sharing  goal  statements)  is  important  (Fielding,  1999). 
MacDonald  (2008)  reports  two  key  components  for  successful  professional  experiences  within  the  CoP.  First, 
members  must  make  an  intentional  effort  to  become  part  of  the  community  and  secondly,  genuine  relationships 
must  be  formed  and  reinforced.  Personal  identity  is  preserved  as  learners  work  within  an  inclusive  environment 
of  equality  and  positive  interpersonal  relationships.  Being  included  and  engaged  are  also  key  to  success  within  a 
community  of  practice  (Wenger,  1998).  Based  on  reports  in  the  literature,  use  of  digital  social  networking  tools 
could  provide  the  authentic  supportive  environment  required  for  successful  21st  century  communities  of 
practice. 


Data  Collection. 

Design-based  research  provides  options  for  a variety  of  methodologies  for  data  collection  (Brush  & Saye, 
2008;  Wang  & Hannafin,  2005).  Several  data  sources  were  used  to  evaluate  the  design  of  our  program’s  internship 
requirements — 1.  analyses  of  master’s  programs  in  educational  technology  requiring  internship  as  degree 
requirement;  2.  analyses  of  discussion  threads  and  reflective  writings;  and  3.  analyses  of  final  reports.  The  portfolios 
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requirements  for  the  degree  include  a final  report,  mapping  activities  with  state  and  national  standards.  We  began  the 
analyses  by  reviewing  online  documents  to  determine  most  common  characteristics  among  educational  technology 
programs.  Program  websites  ( n = 14  ) were  reviewed  and  analyzed  for  commonalities  in  requirements  for  the 
professional  internship.  Programs  were  selected  based  on  accreditation  (NCATE)  with  specialized  professional 
associations  (SPA)  accredited  through  the  Association  of  Educational  Communications  and  Technology  (AECT) 
and  the  International  Society  of  Instructional  Technology  (ISTE).  AECT,  as  a professional  organization,  was 
selected  because  of  the  broad  perspectives  in  professional  programs  in  technology  (i.e.,  instructional  design, 
instructional  technology  in  K12  environments,  and  school  library  media).  ISTE  was  selected  based  on  national 
prominence  related  to  computer  literacy  standards  in  K12  education.  Standards  associated  with  both  AECT  and 
ISTE  are  available  through  the  NCATE  website.  A list  of  all  advanced  programs  with  NCATE  accreditation  is  also 
available.  Selected  programs  associated  with  NCATE  were  identified  and  a review  of  documents  accessible  through 
websites  was  conducted.  Characteristics  of  the  various  elements  were  organized  into  three  categories.  Table  1 
displays  elements  of  internships  in  14  institutions  and  organized  by  category. 

Table  1.  Common  Elements  in  Professional  Programs  in  Educational  Technology. 

Design  of  Graduate  level  Seminar  Assessment  & Evaluation  Theory  to  Practice 

Graduate  credit  course;  1 to  3 credit  Articulate  relationship  between  Standards  aligned  with  theory 

hours  internship  experiences  and  professional  base 

standards 

Predetermined  requirements  for  Internship  proposal  includes  clearly  Reflective  writings  to  match 

practicum  hours  defined  goals  and  objectives  theory  to  experiences; 

standards  to  experiences 

Regular  discussion  with  other  interns  ePortfolio  [or  similar  culminating  Portfolio  products  and/or 

(community  of  learners)  product]  or  presentation  of  work  research  paper 

samples 

Progression  of  simple  to  complex  Sustained  feedback  from  university  Opportunity  to  apply  theory  to 

activities  performed  onsite;  Internship  supervisor  and/or  onsite  mentor  authentic  work  setting 

leads  to  job  offer. 

Experiences  as  instructional  leaders 
[planning  and  implementation] 

Experiences  in  design  of  teacher 
professional  development  (K12  sites) 


The  design  of  our  internship  seminar  was  compared  with  common  elements  identified  in  the  14 
professional  programs  selected  for  this  study.  The  results  showed  correspondence  between  each  of  the  elements  in 
Table  1 with  the  design  of  our  internship  requirements.  Using  the  elements  identified  in  Table  1,  our  faculty 
continually  evaluate  the  design  of  our  online  seminar  course.  In  Table  2,  see  the  design  of  the  seminar  covering  the  3 
semester  hour  credits. 
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Table  2.  Timeline  and  requirements  for  Internship  Seminar  evaluated  for  this  study. 


Major  Requirements 
Timeline  for  3 semester  hour 
graduate  course. 

Total  hours 
for  15-week 
semester  = 
150  contact 
hours 

Aproximate 
Time  per 
week  = 9 
hrs 

Weekly  Activities 

Contact  site  mentor,  goals/ 
objectives,  proposal 

15  hours 

15  hrs  early 
in  the 
semester 

Contact  mentor  first  two  weeks  of  seminar; 
prepare  written  goal  statements;  plan  activities 

Discussion  with  fellow 
interns 

15  hours 

1 hr 

Online  Discussion  threads  connecting  internship 
experiences  with  assigned  readings. 

Assigned  Readings 

30  hours 

2 hrs 

Selected  theory-based  readings  aligned  with 
course  objectives,  standards. 

Reflection  and  writing 

15  hours 

1 hr 

Time  logs  and  reflective  journaling 

Time  applied  planning, 
preparation,  implementation, 
evaluation 

60  hours 

4 hrs 

Meetings,  shadowing,  workshop,  preparation, 
design  of  classroom  instruction,  etc. 

Portfolio  product  with  final 
report  - ongoing 

15  hours 

1 hr 

Maintain  time-log  and  journals  to  document  start- 
end  dates;  description  of  activities. 

End  of  course  surveys,  informal  interviews,  and  students’  final  reports  suggest  interns  are  able  to  complete  1 10 
hours  of  practicum  experiences  in  addition  to  online  discussion,  readings,  and  other  supporting  activities  for  the 
seminar. 


Discussion  Threads  & Reflective  Writings:  Theory  to  practice. 

Research-based  design  methods  permit  the  researcher  to  use  reporting  from  practitioners  in  complex 
settings  to  refine  current  theories  in  education  (Sandoval  & Bell,  2004).  Innovative  learning  environments  can  then 
become  natural  laboratories  “to  study  learning  and  teaching”  (p.  200).  Interns  were  divided  into  six  small  groups  in 
2 course  sections.  Discussions  were  managed  using  Blackboard  Small  Group  Discussion  Forum.  Each  group  was 
assigned  identical  readings  and  prompts  for  discussion  questions.  Questions  were  designed  to  probe  thinking 
processes  as  these  connect  activities  conducted  at  the  school-site  with  readings  and  theory  presented  in  earlier 
courses.  The  response  in  Figure  1 is  taken  from  the  third  of  four  assigned  discussion  forums.  In  addition  to 
discussion  threads,  interns  posted  anecdotal  comments  and  reflections  for  the  final  report  and  formal  reflective 
writings  for  the  final  portfolio  required  for  the  degree  program.  In  Figure  1 see  intern’s  response  to  a discussion 
scheduled  early  in  the  internship.  The  student’s  comment  is  focused  on  the  role  of  the  technology  facilitator  in  kl2 
schools  and  what  is  needed  to  have  impact  on  student  learning. 


44 


Figure  1.  Discussion  Forums  for  Interns. 


Question  Prompt — 

Meet  with  small  group  Jan  26-31.  Have  you  observed 
TF  performance  in  your  school  system  that  matches  the 
national  standards?  Choose  from  any  one  of  the  VIII 
categories  to  discuss.  How  can  you  overcome  obstacles 
for  implementing  TF  Standards  in  your  school  or 
district?  Read  the  comments  of  others  and  offer 
solutions  based  on  the  readings  for  Quality  PD  and 
Effect  of  PD  on  Learning. 


Discussion  Threads — 

Time  management  is  probably  one  of  the  biggest 
obstacles  to  tackle.  Trying  to  maintain  a healthy  balance 
of  instruction  and  administrative  tasks  can  at  times  be 
overwhelming.  ...  I think  it  is  important  to  keep  the 
needs  of  the  students  at  the  forefront.  Instruction  should 
always  come  first.  Planning  a schedule  and  keeping 
working  list  of  goals  helps  to  keep  you  on  task  and  also 
helps  you  to  see  your  accomplishments.  It  also  helps 
you  to  have  focus.  ...  I also  think  you  have  to  accept 
the  fact  that  everything  is  not  going  to  be  perfect  at  any 
given  time,  but  by  having  a focus,  and  goals 
(instructional  and  administrative),  you  can  have  an 
effective  and  efficient  learning  environment.  . . . 
Lawless  and  Pellegrino  state  that  it  is  not  necessarily  the 
“technology  that  improves  education,  but  that 
educational  improvement  comes  through  coherent 
instruction  and  assessment  that  supports  high-quality 
student  learning.”  . . . 


Using  a constant  comparative  method  (Merriam,  2001),  we  analyzed  the  300  paragraphs  of  data  collected 
from  63  interns  hand  coding  them  for  recurring  themes.  We  administered  a word  frequency  count  to  look  for  clusters 
of  words  relevant  to  recurring  themes  identified  in  the  professional  literature.  Comments  in  the  discussion  threads 
included  references  to  well  known  theorists  and  models  in  the  field  of  instructional  design  and  technology. 
Theoretical  constructs  from  earlier  courses  were  reviewed  during  online  discussions.  Interns  were  prompted  to  make 
the  connection  between  experts  in  instructional  technology  with  what  they  were  experiencing  in  the  day-to-day 
interactions  with  teachers,  administrators,  and  students.  For  example,  one  intern  comments- 

“. . .one-on-one  collaboration  between  a subject  matter  expert  (teacher)  and  a technology  expert  (TF)  is 
needed  on  a regular  basis.  Most  training  is  a demonstration  in  a group  setting. . .which  does  not  get  the  same 
results  as  two  people  sitting  together  and  planning  an  lesson,  doing  a collaborative  lesson  together. ...  By 

following  this  model,  I have  the  opportunity  to  impact  student  learning Effective  PD  involves 

participants  in  actual  hands-on  work  in  the  content  area,  covers  enough  hours  to  inspire  changes  in  teaching 
style,  and  provides  a forum  for  peer  discussion  and  assistance.  . . .it  is  extremely  important  to  assess 
incoming  proficiencies  (of  teachers).  ...  I think  the  Lawless  and  Pellegrino  article  would  be  interesting 
reading  for  any  administrator. ...” 

This  student  didn't  identify  a particular  theorist;  however  she  clearly  is  grounding  her  techniques  with  established 
research  in  instructional  design  and  change  theory  - 

“I  have  surveyed  my  staff  and  are  using  the  responses  from  the  survey  to  plan  my  instruction  and  sessions 
offered.  . . I will  offer  some  sessions  for  those  who  are’ innovators’,  some  for  ‘early  adopters’,  and  even 
sessions  that  I think  will  appropriate  for  the  "laggards".  I want  to  try  and  meet  the  teachers  where  they  are. 

I will  do  my  best  to  differentiate  the  instruction  when  needed.  I always  focus  much  of  the  professional 
development  on  how  it  can  specifically  be  used  in  their  curriculum/classes.  “ 


Comments  such  as  these  suggest  intern’s  ability  to  recall  lessons  in  instructional  design,  collaboration,  mentoring, 
and  coaching  as  an  effective  method  for  staff  development.  In  addition  interns  know  it  is  important  to  track  and 
measure  impact  of  teachers’  staff  development  on  student  performance  (Creasy,  2005;  O'Connell  & Phye,  2005). 
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Teachers  need  just-in-time  mentoring  and  follow-up  with  use  of  technology  for  classroom  instruction  (Lawless  & 
Pellegrino,  2007).  Planning  and  implementation  of  staff  development  is  an  important  function  of  technology 
specialists  in  North  Carolina  schools  (North  Carolina  Department  of  Public  Instruction,  2009).  The  technology 
facilitator,  at  the  building  level,  is  able  to  apply  knowledge  and  skills  for  instructional  design  to  the  development  of 
workshops  and  other  formats  for  teachers'  professional  development.  Many  of  the  discussion  threads  suggest 
extensive  experiences  in  planning  staff  development — 

“ our  staff  does  not  know  what  their  training  needs  actually  are.  They  either  think  everything  is  fine  as  it 

is,  or  they  view  training  as  a commitment  to  something  else  to  do. . .Needs  assessments  are  usually 
considered  a required  precursor  to  any  staff  development  project. . . Based  on  last  year’s  test  score  we  need 
staff  development  that  will  help  students  increase  their  reading  and  math  academic  level.” 

The  use  of  state  and  national  standards  provide  guidelines  for  ensuring  interns  are  meeting  competencies 
required  for  licensure.  Internship  activities  are  aligned  with  recommended  standards  for  the  position  of  technology 
coordinator  or  facilitator.  In  Table  3,  four  reflective  writings  provide  evidence  for  projects  that  reflect  future  job 
activities.  The  interns  prepare  written  reports  that  clearly  describe  the  correspondence  between  the  internship  and 
professional  standards.  Thus,  reflective  writings  are  both  theory  and  standard  based  discussions  of  the  internship 
experience. 

Table  3.  Internship  experiences  matched  to  state  and  national  standards  for  technology  facilitators. 


(Intern  1 - Staff  Dev.)  Throughout  the  entire  internship, 
I observed,  shadowed,  and  asked  the  advice  of  one  of 
our  county  technology  facilitators.  . . one  of  the  projects 
that  I have  taken  the  most  from  is  the  actual 
development  of  our  staff  development  module.  We  had 
the  opportunity  at  the  beginning  of  the  school  year  to  be 
a part  of  a series  of  10  technology  related  staff 
developments.  . . We  based  our  module  off  of  the 
ADDIE  module  of  instructional  design  in  order  to 
follow  a structured  plan. 

(Intern  2-  Computer  Skills  tests)  The  NC  Standard 
Course  of  Study  for  integrating  technology  is  developed 
for  all  grade  levels,  K-12.  . . .The  more  exposure  they 
receive,  the  more  their  students  will  benefit  (ISTE 
Standard  III,  Indicator  A), 

(Intern  3-  reflective  practitioner)  . . . having  the 
opportunity  to  reflect  on  each  of  my  experiences  has 
instilled  in  me  the  importance  of  looking  at  education  in 
new  ways  and  the  significance  of  becoming  a reflective 
practitioner. 

( Intern  4-  going  green)  Each  experiences  in  this 
internship  has  given  me  the  opportunity  to  showcase  my 
skills  as  an  instructional  technologist,  but  also  really 
help  my  school  community  by  offering  “go  green”  ways 
to  collaborate,  and  enhance  instruction.  Now,  many  of 
my  colleagues  talk  freely  about  “going  paperless” 
because  of  some  of  the  applications  I have  shared  that 
they  are  now  comfortable  using  independently.  This  has 
been  a great  experience  and  avenue  for  me  to  help  those 
present  in  the  21st  Century  to  actually  start  living  in  the 
21st  Century. 


ISTE  standards 

II.  Planning  and  designing  learning  environments  and 
experiences.  Educational  technology  facilitators  plan, 
design,  and  model  effective  learning  environments  and 
multiple  experiences  supported  by  technology. 

A.  Design  developmentally  appropriate  learning 
opportunities  that  apply 

technology-enhanced  instructional  strategies  to  support 
the  diverse  needs  of  learners. 


A - Facilitate  technology-enhanced  experiences  that 
address  content  standards  and  student  technology 
standards. 


TF-V.  B.  1.  Continually  evaluate  and  reflect  on 
professional  practice  to  make  in  formed  decisions 
regarding  the  use  of  technology  in  support  of  student 
learning. 

TF-VIII  Leadership  and  Vision 

Educational  technology  facilitators  will  contribute  to  the 
shared  vision  for  campus  integration  of  technology  and 
foster  an  environment  and  culture  conductive  to  the 
realization  of  the  vision. 
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Interns  are  instructed  to  prepare  a final  portfolio  with  reflective  writings  that  justify  activities  and  projects  as 
representative  of  competencies  recommended  by  state  and  national  accrediting  agencies. 


Discussion  Threads:  Community  of  Learners. 

Interns  recognized  the  characteristics  and  benefits  of  forming  a community  of  learners.  Though  not  a 
formal  Community  of  Practice  (Lave,  & Wenger,  1991),  there  was  transfer  of  understanding  and  the  formation  of 
professionals  who  participate  in  knowledge  sharing,  mutual  respect,  and  supportive  relationships.  One  Intern 
responds  to  the  idea  of  another  intern's  PD  plans.  In  response  to  a PD  plans  for  designing  teacher  websites,  an  intern 
offers  the  following  support  to  his  fellow  intern: 

“I  also  think  you  touch  on  a very  important  point  and  that  is  reaching  out  beyond  the  walls  of  the  school  to 
the  greater  community  around  them. . ..thanks  for  reminding  me  that  we  share  the  same  seed-planting 
approach  to  change.” 

Use  of  discussion  threads  seems  to  provide  a safe  environment  for  personal  reflection.  As  part  of  the  self- 
assessment  process,  one  intern  reflects  on  the  value  of  workshops  and  other  professional  development  activities 
planned  during  his  internship. 

“This  ..experience  represents  my  future  goals  and  plans  because  it  allows  me  to  understand  and  become 
more  confident  as  an  instructional  designer  and  teacher.  I have  experienced  positive  experiences  with 
workshops  and  negative  experiences  with  workshops.  I was  able  to  revise  and  re -teach  in  order  to  succeed 
as  an  instructor  and  succeed  with  teaching  the  content  for  learning  and  comprehension  of  the  learners.  I 
judged  the  success  of  the  workshops  through  learner  (teachers  ) participation,  conversations  and 
discussions,  along  with  follow-up  assistance  in  developing  and  designing  lesson  plans  and  assistance  with 
facilitating  the  lessons.  As  I strive  to  become  a better  teacher  this  . . .experience  has  helped  guide  my 
success  in  my  future  goals.” 


Discussion  and  Recommendations. 

Partnerships  and  Community  of  Practice. 

A 15  week  semester  was  the  period  of  review  for  this  study.  Two  cohorts  successfully  completed  internship 
requirements  for  an  advanced  degree  program.  Using  Blackboard  conferencing  software,  the  course  timeline  was 
clearly  displayed  throughout  the  semester.  Interns  commented  on  the  importance  for  having  a virtual  meeting  place, 
“The  Hallway”,  to  track  their  own  progress  through  the  semester.  In  an  online  environment,  it  is  important  to 
maintain  consistent  and  frequent  contact  using  email,  discussion  threads,  or  other  interactive  environments.  Because 
of  the  number  of  interns  (n=63)  we  discovered,  through  feedback  and  students’  reflective  writings,  the  importance 
for  articulate  and  authentic  responses.  Just  as  with  face  to  face  dialog,  students  value  guidance  that  speaks  to  his  or 
her  specific  experiences.  This  kind  of  dialog  is  facilitated  through  virtual  environments,  is  beneficial  for  interns,  but 
requires  careful  time  management,  and  dedication,  by  the  university  faculty.  In  many  ways,  online  seminars  exceed 
traditional  face  to  face  courses  by  using  digital  tools  to  form  a supportive  learning  community  leading  to  an 
established  Communities  of  practice  (CoP).  Online  discussions  provided  evidence  that  interns  support  each  other 
with  positive  reinforcement,  provide  useful  feedback  on  projects,  and  share  ideas  for  instructional  development. 
These  discussions  were  prompted  by  the  instructor,  but  often,  comments  are  spontaneous  and  seem  to  flow  from 
relationships  developed  through  early  discussions  used  to  build  social  relationships. 

The  internships  resulted  in  strong  partnerships  with  rural  school  districts  and  the  local  Director  of  Technology. 
During  this  study,  33  local  schools  partnered  with  the  university  to  support  placement  of  interns.  Through  similar 
partnerships  staff  development  workshops,  one-to-one  mentoring,  and  other  forms  of  professional  development 
might  be  possible  at  school  sites  used  for  the  internships.  Reports  through  discussion  threads  and  personal  email 
described  valuable  experiences  in  planning  and  presenting  special  sessions  in  a local  “technology  conference”. 
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Motivation  for  both  teachers  and  presenters  was  high  as  interns  worked  with  the  local  school  system  to  present  their 
best  practices  for  technology  in  the  classroom.  Internships,  planned  collaboratively  with  local  school  administrators 
can  result  in  a variety  of  professional  development  experiences  for  teachers  and  interns  at  the  internship  site.  Several 
important  concerns  were  reported  as  a result  of  dialog  between  administrator  and  interns.  For  example,  planning  for 
professional  development  for  teachers  was  driven  by  purchasing  decisions  rather  than  instructional  need.  Content  for 
staff  development  was  often  determined  by  purchases  for  new  hardware  and  software.  Interns  applied  readings  in 
research  in  professional  development  to  justify  proposals  that  included  a formal  needs  assessment  as  part  of  the 
planning  needed  for  staff  development  workshops.  Discussion  threads  also  described  follow-up  activities  with 
teachers  who  implemented  integration  strategies  presented  in  workshops.  Working  with  administrators  in  the  district 
offices  led  to  new  awareness  in  the  role  of  the  technology  facilitation  as  the  designer  of  professional  development 
for  teachers. 

Theory  to  Practice. 

Discussions  revealed  internship  activities  included  references  to  a theoretical  base  from  courses  completed  in 
the  degree  program.  An  important  goal  for  the  discussion  threads  might  be  forming  a habit-of-mind  that 
substantiates  instructional  decisions  through  use  of  educational  research.  As  interns  model  thinking  processes  that 
connect  theories  related  to  instructional  design  and  technology  to  a classroom  setting,  teachers  in  the  field  might 
benefit  through  a natural  transfer  from  theory  to  practice.  Ongoing  and  online  dialog,  facilitated  by  the  university 
professor,  might  provide  a type  of  laboratory  for  developing  these  kinds  of  discussions.  Ongoing  input  and 
thoughtful  guidance  by  the  university  professor  is  critical  for  successful  online  discussion.  (Berge,  1995;  Paw  an, 
Paulus,  Yalcin,  & Chang,  2003).  Both  motivation  to  participate  and  development  of  good  writing  skills  are  related  to 
input  from  the  course  instructor  ( Dennen  & Wieland,  2007).  The  successful  use  of  question  prompts  with  the 
online  mentoring  further  substantiates  the  results  of  earlier  studies  on  the  role  of  mentors  for  online  internships 
courses  ( Hew,  & Knapczyk,  2007). 

Discussion  threads  provided  evidence  to  show  interns  connect  national  standards  related  to  instruction  and 
learning.  The  standards  most  frequently  referred  to  Technology  Operations  and  Concepts,  and  Teaching,  Learning, 
and  the  Curriculum.  This  would  be  a logical  outcome  for  the  discussions  since  interns  are  required  to  provide 
technical  support  and  in  addition  to  mentoring  for  teachers.  Much  of  the  intern’s  time  is  devoted  to  inventory  and 
organization  of  electronic  resources  in  the  school.  The  growing  trend  for  technology  facilitators  to  become  teacher 
leaders  in  use  of  technology  for  teaching  is  a hopeful  indication  that  learning,  cognition,  and  instructional  design 
take  precedence  over  technical  support. 

Professional  internships  described  in  the  study  would  add  credibility  to  the  technology  facilitator  position. 
Becoming  credentialed  as  an  instructional  leader  is  also  an  important  characteristic  leading  to  prestige  and  respect 
for  this  important  position  in  K12  schools.  The  Internship  is  an  important  stepping  stone  from  the  university  (virtual 
or  actual)  to  authentic  experiences  in  the  workplace.  Other  programs  might  apply  some  of  the  elements  reported  in 
this  study  to  revise  and  enhance  internship  experiences  monitored  in  an  online  environment. 
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Abstract: 

A graduate  course  introducing  in-service  middle  level  mathematics  teachers  to  appropriate  technologies  began 
initially  as  a broad  survey  course.  Based  on  feedback  from  teacher-participants  and  instructor  reflections,  the  course 
was  modified  to  dedicate  more  time  to  fewer  technology  tools.  In-depth  time  was  spent  on  learning  to  use  and  teach 
with  LEGO  Robotics,  Geometer’s  SketchPad,  and  spreadsheets.  The  benefits  and  limitations  of  making  this  change 
are  presented. 

Introduction 

It  is  not  every  day,  or  year  for  that  matter,  that  university  faculty  get  to  develop  new,  from  scratch,  courses. 
Resources  are  generally  limited  and  programs  are  typically  set  in  place  with  defined  curricula.  So,  when  such 
opportunities  do  arise,  lessons  can  be  applied  to  and  learned  from  designing  and  implementing  instruction. 

Such  was  the  case  when,  in  2002,  the  University  of  Wyoming  set  about  designing,  testing  and 
institutionalizing  a set  of  eight  new  courses  to  specifically  prepare  teachers  for  teaching  mathematics  at  the  middle 
school  level,  grades  5-8.  This  effort  was  in  direct  response  to  urging  by  the  National  Council  of  Teachers  of 
Mathematics  (NCTM),  the  Conference  Board  of  the  Mathematical  Sciences  (CBMS),  and  the  National  Science 
Foundation  (NSF)  for  specific,  appropriate  preparation  in  mathematics  content  and  instructional  strategies  for  in- 
service  middle  school  teachers  whose  formal  preparation  is  either  in  elementary  or  a different  content  discipline. 

The  need  for  special  preparation  of  teachers  for  middle  school  mathematics  is  forcefully  expressed  in  the 
updated  NCTM  standards  (NCTM,  2000).  In  addition,  the  CBMS  specifically  recommends  that  students  should  be 
taught  by  “mathematics  specialists”  beginning  in  grade  5 (CBMS,  2000).  Middle-level  teachers  are  often  “caught  in 
the  middle"  in  terms  of  their  professional  preparation.  They  typically  complete  courses  in  either  elementary  or 
secondary  education,  without  coursework  and  professional  development  programs  designed  specifically  for  working 
with  middle-level  learners.  As  stated  in  the  NCTM  standards,  teachers  require  knowledge  that  “is  beyond  what  most 
teachers  experience  in  standard  pre-service  mathematics  courses  in  the  United  States”  (2000,  p.  17).  They  also  need 
specific  knowledge  and  skills  that  are  not  included  in  secondary  programs,  including  adolescent  development, 
pedagogical  alternatives,  and  interdisciplinary  approaches  to  teaching.  As  a result,  NCTM  called  for  special 
attention  “to  the  preparation  and  ongoing  professional  support  of  teachers  in  the  middle  grades.  Teachers  need  to 
develop  a sound  knowledge  of  mathematical  ideas  and  excellent  pedagogical  practices  and  become  aware  of  current 
research  on  students'  mathematics  learning”  (2000,  p.  213). 

Specific  objectives  of  this  project,  funded  by  an  NSF  middle-level  mathematics  program  grant,  were  toil) 
Increase  the  conceptual  content  knowledge  of  middle-level  teachers;  2)  Increase  the  pedagogical  content  knowledge 
of  middle-level  teachers;  and  3)  Provide  middle-level  teachers  experiences  with  exemplary  middle-level 
mathematics  curricula  and  help  them  to  develop  skills  in  implementing  them  in  their  own  classrooms.  The  resulting 
courses  were: 

• Connecting  Geometry  with  Problem  Solving  for  the  Middle-Level  Learner; 

• Assessment  for  Middle-Level  Mathematics; 

• Mathematics  of  Change  and  the  Middle-Level  Learner; 

• Numbers,  Operations,  and  Patterns  for  the  Middle-Level  Learner; 

• Analysis  of  Data  in  the  Media  for  the  Middle -Level  Learner; 

• Social  and  Historical  Issues  in  Mathematics  and  Middle-Level  Learner; 

• Methods  for  Teaching  Middle-Level  Mathematics;  and 

• Using  Instructional  Technology  for  Middle-Level  Mathematics. 

The  Instructional  Technology  course  focused  on  introducing  in-service  middle-level  teachers  to  technology  tools 
which  could  be  used  to  enhance  instruction  in  meaningful  and  creative  ways. 

This  course,  taught  in  a three-week  period  in  the  summer,  was  initially  designed  as  a survey  course,  and  as 
such,  introduced  a broad  range  of  hard  and  soft  technologies.  Technologies  were  selected  which  would  be  student- 
centered  and  student-directed,  enabling  them  to  solve  complex  and  ill-defined  real-world  problems.  In  the 
exuberance  and  excitement  of  developing  such  a course,  with  the  desire  to  introduce  teachers  to  a breadth  of  tools. 
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an  aggressive  schedule  was  set  involving  11  technologies  (See  Fig.  1).  These  included  graphing  calculators, 
Geographic  Information  Systems  (GIS),  Global  Positioning  System  (GPS)  units,  geometry  microworlds,  several 
spreadsheet  programs,  LEGO  robotics,  and  Vernier  probeware.  Course  sessions  focused  on  activities  and  lessons 
designed  and  used  by  other  middle-level  math  teachers,  who  actually  team-taught  the  course. 
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Fig.  1:  Course  Schedule  for  First  Iteration 


Changes  in  Course  - data  used  Revised  Technology-intensive  Survey  Course 

After  the  initial  course  was  taught  in  Summer  2003,  data  were  collected  in  Spring  2004  to  ascertain 
whether  teachers  found  the  technology  useful  or  not  useful  in  their  teaching,  and  to  find  out  what  was  used  and  what 
they  would  have  liked  to  leant  more  about.  Subsequent  survey  data  from  the  teacher-participants  showed  that  the 
course  was  helpful  in  exposing  them  to  possibilities,  but  did  little  to  prepare  them  for  actually  integrating  any  of  the 
technologies  into  their  classrooms.  The  participating  teachers  were  simply  not  given  sufficient  time  to  become 
familiar  with  the  tools.  One  participating  teacher  made  the  poignant  comment:  “Whatever  we  did  in  class  last 
summer,  I could  not  make  it  work.  I felt  as  though  I'd  been  dumped  into  the  middle  of  a large  lake  and  just  told  to 
swim  home.  It  was  sink  or  swim.  I sank."  Participants  also  expressed  a desire  to  learn  more  about  calculators, 
spreadsheets.  Geometer’s  Sketchpad  (GSP),  and  Fathom,  a database/spreadsheet  program  written  specifically  for 
middle  grade  use.  They  felt  these  would  be  useful  in  their  teaching. 

The  second  iteration  of  the  course,  taught  in  Summer  2004,  introduced  only  six  technologies  (See  Fig.  2), 
with  an  increased  emphasis  on  Geographic  Information  Systems  (GIS)  software,  namely  Arc  Explorer.  The  potential 
for  integrating  social  studies,  science,  and  mathematics,  including  tabular  and  spatial  data  analysis,  was  seen  as  a 
major  asset  of  the  software.  As  only  two  3.5  hour  sessions  had  been  set  aside  in  the  2003  offering,  four  sessions 
were  scheduled  in  2004  to  provide  more  time  for  deeper  and  richer  instruction,  including  multiple  middle-level 
learning  activities.  Calculator  and  probeware  instruction  was  removed  from  the  Technology  course  and  placed  in  the 
Mathematics  of  Change  course,  with  specific  instruction  as  to  how  to  use  the  technologies  while  doing  the 
mathematics.  This  integrated  approach  was  viewed  as  being  more  beneficial  to  the  teachers  in  the  long-term. 
Additionally,  even  though  GSP  had  been  designated  originally  to  be  introduced  in  the  Geometry  course,  in  practice 
it  was  implemented  at  an  intermediate  level,  leaving  many  students  frustrated  and  confused.  To  rectify  this,  the 
technology  course  was  designated  as  the  initial  teaching  point  for  this  software. 
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Fig.  2:  Course  Schedule  for  Second  Iteration 
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Latest  iteration-Focusing  on  Fewer  Still 

Little  changed  in  the  Summer  2006  iteration  of  the  course,  but  significant  changes  were  afoot.  The 
instructors  were  introduced  to  a newly  published  book.  Meaningful  learning  using  technology:  What  educators  need 
to  know  and  do  (Ashburn  & Floden,  2006).  The  frameworks,  questions,  and  examples  posed  by  the  various  chapter 
authors  were  extremely  helpful  in  reviewing  and  reflecting  on  the  purposes,  goals,  and  outcomes  of  the  course.  More 
specifically,  the  attributes  of  meaningful  learning  using  technology  (Ashburn,  2006;  Wiske,  2006),  the  affordances 
for  learning  and  teaching  (McCrory,  2006),  and  the  conditions/attributes  of  better  professional  development  (Zhao, 
Frank  & Ellefson,  2006;  Floden  & Bell,  2006)  played  key  roles  in  rethinking  the  course.  In  summer  2008  the  course 
was  modified  so  that  only  three  technologies  would  be  introduced:  LEGO  robotics,  GSP,  and  spreadsheets,  with 
instruction  and  learning  focusing  on  the  central  question  of,  “What  is  it  about  this  technology  that  will  support 
students  in  engaging  with  the  big  ideas  of  [mathematics]?”  (McCrory,  2006,  pg.  147). 

Out  of  twelve  3.5  hour  class  sessions,  four  were  devoted  to  robotics,  four  on  GSP,  and  two  on  spreadsheets 
(See  Fig.  3).  Each  was  used  in  a variety  of  activities,  lessons  and  performance  assessments  to  not  only  teach  the  use 
of  the  tools,  but  to  model  appropriate  classroom  use  in  the  middle-grades.  For  example,  middle-level  GSP  activities 
from  Shape  makers:  Developing  geometric  reasoning  in  middle  school  with  The  Geometer’s  Sketchpad  (Battista, 
2003),  and  Geometry  activities  for  middle  school  students  with  The  Geometer’s  Sketchpad  (Wyatt,  Lawrence,  & 
Foletta,  2004)  were  used.  Discussions  were  facilitated  about  the  use  of  the  tools  and  lessons  from  the  participants’ 
dual  perspectives  as  students  in  the  class  and  as  teachers,  to  identify  strengths  and  limitations  of  each  for  enhancing 
learning,  to  explore  strategies  for  incorporating  new  technologies  in  their  current  settings  and  circumstances,  and  to 
share  emerging  ideas  for  modifying  their  curricula  to  include  the  use  of  technology  tools.  Course-based  feedback 
from  the  teachers  was  positive  about  the  course  structure  and  time  spent  on  each  technology  tool.  Even  then,  some 
teachers  stated  that  they  wished  that  they  could  have  spent  more  time  with  each. 
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Figure  3:  Third  iteration  schedule 


Participant  Outcomes 

In  addition  to  the  course,  support  has  been  provided  to  teachers  during  the  2008-09,  09-10  school  years  as  they 
implemented  changes  in  their  teaching.  The  course  instructors  have  made  classroom  visits  to  engage  students  with 
LEGO  Robotics,  have  loaned  robotics  kits  and  laptop  sets  to  teachers  to  use  on  an  extended  basis,  and  have 
responded  to  phone  and  email  requests  for  assistance  in  planning  and  implementation. 

While  formal  data  have  yet  to  be  collected  from  the  latest  teacher  participants,  initial  self-reports  and 
instructor  contacts  indicate  that  the  teachers  are  confident  in  their  preparation,  with  specific  plans  for  integrating  the 
technologies  into  their  instruction.  For  instance,  one  teacher  responded  to  an  assignment  focusing  on  future  plans  by 
stating. 

The  first  area  that  I would  like  to  infuse  new  technology  is  in  my  beginning  of  the  year  team-building 
activities.  We  usually  play  a variety  of  games,  introduce  some  short-term  problem  solving  activities,  and 
try  to  come  to  terms  on  how  our  room  should  operate.  I believe  the  Lego  Robotics  and  Mindstorms 
software  would  be  an  outstanding  way  to  build  community  and  accomplish  procedural  goals  with  a higher 
level  of  student  thinking.  I think  that  this  is  a good  use  of  technology  because  it  lends  itself  so  well  to 
small,  cooperative  groups.  The  immediate  feedback  provided  by  the  robot  to  the  students  is  powerful. 

They  are  in  charge  of  their  robot,  not  an  all  knowing,  all  powerful  teacher.  Therefore,  they  must  take 
ownership  and  deal  with  problems  that  arise.  These  are  early  experiences  that  I,  as  a teacher,  can  use  to 
teach  kids  how  to  think  their  way  to  possible  solutions.  I also  loved  the  notion  of  “gracious 
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professionalism”  as  we  worked  on  our  own  projects.  Kids  need  to  be  encouraged  to  be  kind  to  one  another. 
If  this  expected  kindness  can  be  paired  with  the  communication  of  their  logical  thinking,  then  we  have  a 
more  robust  activity.  Finally,  an  important  tenet  of  my  classroom  is  “Failure  leads  to  success.”  Almost 
invariably  you  have  to  fail  in  order  to  get  feedback.  These  robots  would  enhance  the  learning  of  that 
concept. 

He  has  since  checked  out  the  LEGO  Robotics/computer  sets  twice  to  use  with  his  classes.  His  class  even  had  the 
local  police  department  bring  in  their  explosive  sniffing  robot  to  investigate  similarities  (See  Figure  4)  to  their 
projects.  He  uses  the  robots  in  much  the  same  way  the  2008  course  was  taught,  as  an  open-ended  problem-solving 
set  of  activities. 


Figure  4:  Sixth  grade  students  and  robotics 


Another  participant  initially  wrote. 

In  Trigonometry,  students  work  with  the  Law  of  Sines  when  covering  right  triangle  trig.  To  help 
understand  the  ambiguous  case  of  Angle-Side-Side,  students  are  shown  the  relationship  between  the  given 
sides  and  or  altitude  to  determine  if  the  ambiguous  case  will  produce  2 triangles,  1 triangle,  or  no  triangle. 
This  concept  has  been  very  difficult  to  demonstrate  but  I plan  on  having  the  class  discover  the  relationships 
using  GSP.  The  most  difficult  part  of  infusing  a GSP  activity  will  be  providing  detailed  instructions  on  how 
to  sketch  the  triangles  and  analyzing  the  relationships  between  the  given  sides  and/or  altitude. 

Prior  to  taking  the  course,  the  school  had  the  software  for  several  years  but  it  was  used  infrequently,  if  at  all.  When 
queried  as  to  why,  her  response  was,  “We  don’t  have  access  to  the  computer  lab — that  is  for  English  classes.”  She 
has  since  fought  for  lab  access,  and  now  her  mathematics  department  has  days  scheduled  and  one  can  observe  her 
classes  as  well  as  others  using  the  computer  lab  for  GSP  on  a regular  basis. 

A third  participant  in  the  2008  class  has  used  the  robots  and  computer  sets  with  her  students  three  different 
times  in  the  past  two  years.  This  teacher  used  them  at  the  beginning  of  each  year  and  as  an  afterschool  activity  for  a 
LEGO  Robotics  competition.  She  initially  wrote, 

I currently  introduce  graphing  of  linear  equations  by  hand.  Based  on  the  information  from  class  I plan  to 
use  graphing  calculators  to  introduce  linear  equations.  This  is  a fairly  big  step  for  me  because  prior  to  this 
class  my  approach  has  been  to  teach  the  topic  and  algorithms  then  enhance  with  technology.  I expect  the 
students  to  acquire  a better  concept  of  positive  and  negative  slopes,  as  well  as  be  able  to  visualize  the 
behavior  of  a line  as  the  slope  approaches  zero,  and  what  it  means  for  a line  to  have  an  undefined  slope.  I 
then  plan  to  reinforce  this  concept  with  Geometer’s  Sketchpad  to  have  the  students  display  the  linear 
equation  and  discuss  the  changing  values  of  the  slope  as  they  drag  to  change  the  slope.  Students  will  see 
application  of  slope  using  three  different  modalities.  This  use  of  technology  will  enhance  learning  because  I 
can  alter  the  nature  of  the  engaging  tasks  (McCrory,  2006). 

In  Consumer  Math  I teach  a section  on  buying  a home.  The  students  use  pre-made  amortization  charts  to 
see  payment  schedules  depending  on  the  percentage  rate  and  pay  out  period.  Students  have  difficulties 
understanding  the  idea  that  a larger  portion  of  the  payment  applies  to  interest  initially  and  a smaller  amount 
to  principle.  I am  anxious  to  use  Excel  to  have  them  use  a formula  to  set  up  their  own  amortization  chart  for 
a mortgaged  amount.  The  demonstration  of  the  variable  flexibility  in  Excel  was  extremely  powerful.  I plan 
to  have  them  set  up  the  mortgage  amount,  the  time  in  years  of  the  loan,  and  the  interest  rate  as  variables. 
The  dynamics  of  Excel  will  allow  them  to  see  the  effects  on  payments  when  changing  the  mortgage 
amount,  the  time  of  the  loan,  and  the  interest  rate.  Student  learning  will  be  enhanced  because  they  should 
make  the  connections  quicker  and  at  a deeper  level  from  the  immediate  feedback  of  changing  one  variable 
at  a time  (Bransford,  Brown  & Cocking,  2000). 
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Lessons  Learned 

This  instructional  technology  course  was  planned  for  practicing  teachers  who  did  not  necessarily  have 
experiences  with  technology  for  teaching  middle-level  mathematics  concepts.  For  this  reason,  there  was  no  focus  on 
general  technology,  as  most  teachers  had  familiarity  with  a computer,  calculator,  and  other  general  technologies 
such  as  overheads,  etc.  Rather,  the  course  concentrated  on  software  and  hardware  that  were  useful  in  mathematics 
teaching,  including  spreadsheet  software,  GSP,  LOGO,  GIS,  Fathom,  and  Cabri  and  LEGO  robotics.  The  primary 
goal  was  to  generate  change  in  how  teachers  think  about  integrating  technology  into  their  normal  teaching  and  that 
teachers  would  have  a more  natural  flow  in  considering  what  would  be  appropriate,  and  have  a higher  comfort  level 
in  using  technology  with  their  students. 

In  the  first  year  the  course  was  taught,  multiple  instructors  were  gathered  to  team  teach.  As  a result,  there 
were  up  to  five  instructors  in  the  room  trouble-shooting  technology  problems.  This  level  of  support  was 
unsustainable,  but  has  been  maintained  by  two  instructors  in  subsequent  implementations.  This  type  of  support  has 
been  critical  to  help  fix  equipment,  answer  questions,  and  guide  investigations.  Also,  several  laptops  were  provided 
initially  while  some  teachers  brought  their  own.  This  turned  out  to  be  challenging  as  some  teachers  did  not  have 
access  codes  given  to  them  by  their  technology  administrators,  preventing  the  installation  of  key  software.  In  the 
latest  iterations  a full  set  of  computers  has  been  provided  for  the  class,  to  which  the  instructors  had  full 
administrative  access.  The  participants  are  free  to  check  out  a computer  overnight  for  homework  as  requested. 

It  was  quickly  obvious  that  our  effort  to  teach  all  of  the  technologies  possible  would  result  in  none  of  the 
technologies  being  learned.  Our  efforts  to  focus  on  technologies  which  teachers  found  to  be  the  most  useful  yielded 
in  greater  classroom  implementation.  The  teachers  are  more  comfortable  with  the  technologies  when  they  complete 
the  course,  and  are  more  willing  to  integrate  the  technology  use  in  their  instruction.  In  the  end,  spending  more  time 
on  fewer  technologies  reaped  greater  rewards  in  the  long  run. 
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Chinese  Online  Education  into  New  Millennium 


Xiaojun  Chen 

People’s  Republic  of  China  is  the  most  populated  and  the  third  largest  country  in  the  world.  Located  in 
Southeast  Asia,  China  is  still  a developing  country  because  of  its  low  GDP  and  the  weak  economic  foundation; 
however,  China  has  surprised  the  world  with  its  dramatic  economic  growth  and  technological  advancement  in  the  past 
two  decades.  There  is  a huge  expansion  of  Internet  usage  in  China,  which  subsequently  influenced  Chinese  education 
at  large.  There  were  forerunners  investigating  the  implications  and  impacts  of  Internet  in  China,  however,  at  present 
little  is  known  about  Chinese  online  distance  education  and  its  sustainable  development.  The  purpose  of  this  article  is 
to  discuss  the  recent  changes  that  Internet  and  its  web-based  applications  have  brought  into  Chinese  education  system 
in  higher  education,  K-12  and  professional  training. 

The  author  found  out  that  there  are  urgent  needs  for  Chinese  educators  to  make  important  shifts  when  Internet 
and  its  web-based  applications  have  penetrated  into  the  current  education  system.  These  shifts  are  the  role  change 
from  teaching  to  facilitating  for  normal  pedagogues  and  the  updating  the  traditional  restricting  classroom  to  an  open 
and  self-directing  online  learning  environment.  The  author  also  believes  there  is  a need  for  global  partnership  in 
Chinese  online  education  system,  especially  in  higher  education;  at  the  same  time,  web-based  trainings  can  be  a 
pioneer  in  leading  the  life-long  learning  strategy  in  China. 

Internet  development  in  China  and  in  Chinese  education 

The  first  incident  of  Internet  application  in  China  dated  back  to  September  20th  1987,  when  an  email  was  sent 
out  by  a scientific  working  group  from  China  to  Karlsruhe  University  in  Germany.  The  Email  title  was  “Across  the 
Great  Wall  we  can  reach  every  corner  in  the  world",  which  symbolized  the  new  era  in  Chinese  information 
technology.  In  the  next  ten  years  (1987-1996),  China  experienced  the  initial  stage  of  Internet  implementation.  Four 
major  networks  were  established  in  China  in  this  decade:  China  Network  (ChinaNet),  China  Golden  Bridge  Network 
(ChinaGBN),  the  China  Education  and  Research  Network  (CERNET)  and  China  Science  and  Technology  Network 
(CSTNet)  (Tang,  2000).  China  Internet  Network  Information  Center  (CNNIC)  was  established  in  May  1997.  It  was 
not  until  then  that  these  four  national  Internet  backbones  joined  to  work  together.  In  the  following  four  years 
(1997-2000)  Chinese  Internet  development  was  geared  by  Chinese  Open  and  Reform  policy.  Word  Computer 
Congress  2000  was  held  in  China,  and  it  was  on  this  congress  that  the  Chinese  government  stressed  the  necessity  of 
setting  up  Internet  pact  and  also  called  on  people  to  make  full  use  of  Internet.  Since  October  1997,  CNNIC  has 
published  24  Semiannual  survey  reports  on  the  development  of  China’s  Internet,  reporting  an  increase  of  Chinese 
Internet  users  from  620,000  in  1997  to  26.5  million  by  July  2001  then  to  162  million  by  July  2007  (CNNIC,  1997, 
2001  July,  2007  July).  This  number  has  again  been  doubled  to  338  million  in  July  2009  according  to  CNNIC’s  newest 
survey  (CNNIC,  2009). 

Realizing  the  key  role  of  information  technology  in  national  economic  growth,  the  Chinese  central 
government  has  made  a series  of  changes  in  its  policy  regarding  the  application  of  technology  in  education  since  1993. 
The  Guideline  of  Educational  Reform  and  Development  was  issued  by  the  Central  Committee  of  the  Communist  Party 
of  China  in  1993,  and  shortly  afterward,  in  1995,  the  Educational  Law  of  the  People’s  Republic  of  China  (Zheng, 
Ouyang,  & Feng,  2002).  These  two  significant  policies  increased  government  support  in  distance  learning,  in  the  form 
of  broadcasting,  television,  and  other  information-technology-based  avenues  “to  create  an  environment  for  lifelong 
learning”(Chen,  1999). 

It  was  started  from  the  higher  education  level,  where  the  research  force  was  grouped  together,  that  the 
application  of  Internet  was  integrated  and  online  education  became  popular.  One  of  the  major  networks  in  China  — 
“China  Education  and  Research  Network  CERNET”,  a high-speed  backbone  network  project  (1999 — 2001)  — 
obtained  its  qualification  from  the  government.  The  project  was  a main  element  of  “the  modern  remote  education 
project”  in  “the  national  education  promotion  plan  of  21st  century”,  and  also  an  important  base  for  constructing  the 
lifelong  education  system  in  China.  Based  on  CERNET,  the  Ministry  of  Education  (MOE)  had  approved  four 
universities  to  launch  their  network  educational  institutions  (later  on  67  colleges  were  included  in  the  list),  and  also 
allowed  19  online  cooperative  research  centers  to  carry  out  remote  education  and  scientific  research  through  CERNET 
("Chinese  Modern  Remote  Education",  2002;  CNNIC,  2003).  Internet  was  also  implemented  in  university 
administrative  systems  in  China.  In  1999,  over  200  colleges  in  six  provinces  used  “All-China  College  Students 
Recruiting  System”  on  CERNET,  and  achieved  its  first  success  (CNNIC,  2001).  It  is  also  reported  that  more  than  300 
higher  education  institutions  and  research  organizations  were  connected  through  these  four  major  networks  across  the 
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country  (Zheng  et  al.,  2002).  This  gives  Chinese  universities  great  opportunities  to  share  research  and  teaching 
resources  at  lower  cost,  without  travel  and  management  expenses.  It  also  helps  to  build  a knowledge  sharing 
foundation  for  pedagogical  and  research  innovation  across  China. 

Online  education  in  China 

When  Internet  was  first  introduced  into  China,  Chinese  people  were  overwhelmed  by  the  abundant 
information  provided  by  Internet  (Zheng  et  al.,  2002).  However,  people  are  getting  more  and  more  comfortable  with 
this  new  type  of  medium  with  the  help  of  the  government's  promotion  and  loose  in  censorship,  which  allows 
self-regulated  media  and  foreign  investment  in  Chinese  telecommunication  services  ("New  Regulations  for  Chinese 
Internet  Use",  2000).  The  most  recent  survey  conducted  by  CNNIC  shows  that  Internet  has  become  the  major  medium 
for  people  to  gain  information,  over  the  traditional  ways  of  TV,  radio  and  newspaper  (CNNIC,  2009).  This  laid  a 
foundation  of  promoting  Internet-based  education  among  Chinese  Internet  users.  Another  survey  conducted  by  a 
Chinese  research  company  I-Research  in  2005  shows  that  75.8%  of  the  experienced  Internet  users,  who  have  a net  age 
above  two,  have  enrolled  in  online  education  or  training  program  (CNNIC,  2006).  Although  the  newest  figure  in  the 
2009  survey  shows  that  the  overall  use  rate  of  online  education  in  2008  was  16.5%,  there  is  a reported  40%  increase 
rate  from  2007  to  2008.  It  is  noticeable  that  with  the  permission  of  government  and  the  availability  of  funding  from 
business  partners,  there  are  three  dimensions  of  Chinese  online  education: 

1)  online  learning  in  higher  education,  based  on  CERNET; 

2)  online  study-help  program  in  Chinese  K-12  program; 

3)  online  work-based  training. 

Online  learning  in  higher  education  - a highway  to  the  future? 

The  Chinese  Ministry  of  Education  started  to  fund  distance  online-learning  programs  for  Chinese  research 
universities  to  participate  in  this  innovative  way  to  enable  education.  Four  universities  (Tsinghua  University,  Beijing 
Posts  and  Telecommunication  University  from  Northern  China,  Hunan  University  from  Middle-south  China  and 
Zhejiang  University  from  Eastern  China)  among  "211  projects  " (TOO  Key  and  Research  Chinese  universities  for  21st 
century)  were  the  pioneers  of  educational  technology  in  China.  These  universities  began  to  research  on  the  design  of 
various  online  courses  and  the  integration  of  current  available  technologies  into  the  conventional  ways  of  teaching 
such  as  the  application  of  multimedia  into  the  traditional  college  classroom  teaching.  Though  these  universities 
shared  the  common  vision  to  enhance  Chinese  higher  education  through  Internet-based  technological  education,  they 
had  different  priorities.  Tsinghua  University  focused  on  web-leaming  instructional  design;  Hunan  University  and 
Zhejiang  University  focused  on  increasing  the  accessibility  of  educational  resources  among  universities  and 
promoting  community-based  learning  in  higher  education  institutions.  The  outcomes  were  very  positive.  Tsinghua 
University  launched  its  own  online  learning  platform  in  2002,  while  Hunan  University  established  its  Distance 
Learning  College  (DLC)  in  2001.  Distance  Learning  College  in  Hunan  University  was  one  of  the  earliest  institutions 
offering  university  degrees  for  online  enrollers.  These  educational  progresses  had  favorable  influence  in  Chinese 
economy,  politics  and  community  life.  There  was  a reported  increase  in  the  number  of  online  course  users  and  more 
universities  started  online  programs  and  online  courses,  such  as  NanJing  University  and  Zhongshan  University  and 
etc.  Some  international  IT  companies  were  attracted  to  educational  technology  in  China  as  well.  In  2003,  Intel  signed 
up  an  agreement  with  Chinese  government  to  provide  technology  and  financial  support  to  the  increasing  demand  for 
educational  technology  within  Chinese  universities(CNNIC,  2006). 

At  the  same  time,  criticism  was  aroused  national  wide  about  administrative  issues  of  online  courses, 
especially  online  degree  awarding.  The  nature  of  online  learning,  i.e.  providing  a learning  environment  for  users 
("whoever")  to  have  equal  access  ("wherever")  to  knowledge  at  their  own  time  ("whenever"),  was  put  into  great  doubt 
of  its  credibility.  Although  distance  education  can  be  dated  back  in  1950s  in  China  and  there  were  claims  of  its 
influences(Carr-Chellman  & Zhang,  2000),  however,  it  is  still  be  considered  as  second-class  education.  According  to 
Chinese  culture,  personal  academic  background  plays  a vital  role  in  assessing  individual's  social  status;  in  other  words, 
Chinese  people  sometimes  prefer  traditional  learning  environments  (for  example,  get  a degree  from  a well-known 
university).  Since  2004  Chinese  educational  technology  has  experienced  a great  decline  in  the  higher  education 
setting.  For  example,  Tsinghua  University  halted  on-going  development  of  its  learning  platform.  Furthermore,  some 
extreme  incidents  that  took  place  during  this  period  put  the  online  degree  awarding  institutions  in  an  awkward 
situation.  Hunan  University  ceased  awarding  online  degrees  for  a two-year  period  after  a plagiarism  incident  during 
final  exams  for  an  online  course  ("A  Review  of  Remote  Education  at  Hunan  University,"  2003). 
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Challenges  and  opportunities,  offered  by  Internet  and  its  web-based  application,  are  both  open  to  the  Chinese 
higher  education.  Cultural  influence  on  public  awareness  is  hard  to  be  changed  in  a short  period  of  time.  However, 
with  the  rapid  development  of  technology  and  globalization  in  economy,  China  still  has  a great  and  increasing  need  for 
online  distance  learning.  With  the  foreign  investment  in  telecommunication  and  education,  and  the  partnership  with 
foreign  education  institutions,  Chinese  online  degree  program  has  more  opportunities.  Since  education  abroad  at 
specific  institutions  has  a reliable  and  good  reputation  in  China  (Carr-Chellman  & Zhang,  2000;  Zhang,  2005),  as  far 
as  the  author  concerns,  Chinese  higher  education  institutions  may  consider  to  take  a less  traveled  road  to  develop 
online  education,  that  is  — to  make  partnership  (with  government  permission,  of  course)  with  foreign  universities  to 
offer  online  courses  or  even  degrees.  On  January  12,  2004,  the  three  nations’  project  (China-US-Russia)  - “Global 
Ring  Network  for  Advanced  Applications  Development”  (GLORIAD)  was  accomplished.  GLORIAD  was  funded  and 
constructed  by  CAS,  the  National  Science  Foundation  (US)  and  Russia  Ministry  and  Science  Group  Alliance. 
GLORIAD  will  support  the  three  countries  even  the  global  advanced  scientific  and  educational  applications  (CNNIC, 
2006).  The  initiative  of  this  project  can  be  used  as  foundation  of  global  educational  partnership.  The  potential 
development  of  global  partnership  can  provide  more  resources  to  both  teachers  and  students  in  higher  institutions.  On 
the  other  hand,  the  pedagogical  shift  from  exam-driven  learning  environment  to  more  self-directed  learning 
environment  is  needed  in  Chinese  higher  education  system.  It  is  the  drive  force  in  sustainable  development  of  Chinese 
online  education,  the  highway  to  the  future. 

Online  study-help  in  Chinese  K-12  education — a safe  way  to  the  future? 

Similar  to  the  development  of  online  program  in  higher  education,  there  is  an  increased  use  of  web-based 
application  and  program  at  school  level.  As  compulsory  education  in  China,  Chinese  K-12  Education  has  a large 
potential  to  make  use  of  Internet  and  its  web-based  applications  in  learning  and  teaching. 


Figure  1 Education  Structure  of  Chinese  Internet  Users 

As  shown  in  Figure  1,  CNNIC  2009  July  survey  reports  that  nearly  75%  of  the  Chinese  Internet  users  are 
from  high  school  or  below  level  of  education.  This  trend  has  aroused  attention  of  government,  schools,  and  business. 
According  to  CNNIC,  the  Chinese  K-12  Education  has  seen  a favorable  progress  in  integrating  Internet  into  daily 
schooling  in  21st  century.  In  2001,  the  national  Internet  project  of  “Every  School  Accesses  the  Network”  entered  the 
phase  of  formal  implementation;  and  “the  Civilized  Internet  Pact  of  All  Country  Teenagers”  was  presented  to  regulate 
millions  of  domestic  teenagers’  behaviors  when  using  Internet  (CNNIC,  2003).  SOHU  and  SINA,  the  two  big  Chinese 
portrait  website  started  their  Net-School  targeting  at  K-12  level  from  2000  and  2001  respectively.  ChinaEdu.com,  a 
commercial  website  providing  online  study  help  session  for  K-12  students,  ranked  top  10  hits  website  in  2006. 
However,  not  all  progress  resulted  in  positive  educational  value.  CNNIC  2007  survey  shows  students  use  Internet  at 
the  order  of  following:  recreational  tool  communication  tool>  information  channel>  life  helper  (CNNIC,  2007 
January).  The  students  mainly  use  Internet's  recreational  function,  even  information  function  comes  next.  This 
phenomenon  should  come  into  notice.  It  is  necessary  to  provide  more  education  and  guidance  to  help  student  know 
more  functions  of  Internet,  especially  in  the  help  of  study. 

Schools  and  education  organizations  were  encouraged  to  develop  Internet-based  instructions  and  design 
relative  materials  for  students  in  the  classrooms  (He,  1998).  Online  study  help  sessions  became  popular  in  big  cities 
and  some  more  developed  provinces  in  China,  e.g.  Beijing,  Shanghai,  Zhejiang,  Guangdong  and  etc.  High  ranked 
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schools  in  these  areas  put  tutorial  programs  online  for  students  to  have  access  at  their  own  time  and  pace.  This 
pedagogical  change  is  a revolutionary  step  in  Chinese  education.  It  calls  out  the  shift  from  classroom  and  exam-driven 
education  to  skill-developing  open  education. 

Digital  divine  is  inevitable  in  China  with  its  unbalanced  regional  economic  growth.  Internet  is  a double  blaze. 
It  deepens  the  digital  divine  at  individual  level,  i.e.  between  students  from  more  developed  areas  and  those  from  less 
developed  parts  of  China.  The  less  developed  parts  of  China  usually  lack  of  technology,  but  more  importantly  lack  of 
educational  resources,  such  as  good  teachers,  and  good  facilities.  Though  the  Chinese  Ministry  of  Education  has 
increased  the  funding  to  support  K-12  education  in  some  remote  rural  areas,  these  efforts  are  far  less  to  bridge  the  gap 
completely.  The  unbalanced  development  rate  challenges  the  quality  of  education  across  the  country.  However, 
Internet  definitely  can  be  served  as  a transmission  vessel  of  knowledge.  Ideally,  with  government  support  policies,  it 
allows  students  to  access  resources  not  available  in  their  classrooms  or  regions;  it  gives  students  more  study-help 
without  time  or  geographical  constrains.  It  can  be  a safe  (promising)  way  to  promote  equal  education  quality 
(opportunity)  across  the  country,  a safe  way  to  the  future. 

Online  work-based  training — an  innovative  way  to  the  future? 

Work-based  training  in  this  article  is  defined  as  training  relating  to  jobs  and  job-related  skills  in  working 
environment,  including  foreign  languages,  specific  IT  skills,  and  etc.  Work-based  training  is  one  of  the  forms  of 
informal  education,  and  has  been  advanced  with  the  help  of  Internet  and  its  web-based  applications  (Overwien,  2000; 
Rossett  & Sheldon,  2001).  Online  work-based  training  is  a new  perspective  to  Chinese  Internet  users,  but  yet  users 
adapted  to  it  very  well.  Job-related  training  and  work-based  training  became  popular  in  China,  as  there  is  an 
increased  profit  in  educational-related  companies.  With  the  development  of  economic  globalization  in  China,  there  is 
a huge  demand  in  foreign  language  learning  and  IT  certificate  training.  These  are  the  two  most  popular  online  training 
programs  in  China  at  the  moment.  The  online  professional  training  program  also  witnessed  an  increased  enrolling 
number  in  the  past  three  years  from  2003  to  2006  (CNNIC,  2006). 


Figure  2.  Age  of  Chinese  Internet  Users 

As  shown  in  Figure  2,  the  newest  survey  report  from  CNNIC  in  July  2009  shows  that  about  65%  of  the 
Chinese  Internet  users  are  aged  below  30.  This  group  of  users  will  dominate  the  development  of  Internet  in  China  in 
the  next  20-30  years.  This  gives  an  indication  of  future  development  leading  to  life-long  education  environment.  The 
work-based  learning  fits  into  the  Chinese  government  education  policy  of  promoting  lifelong  learning  (Chen,  1999), 
i.e.  sustain  people’s  learning  and  education  through  various  ways  of  distance  learning  program.  There  is  a need  to 
promote  online  work-based  training  for  Chinese  young  users  and  to  cultivate  their  life-long  learning  habit  through 
training.  People  no  longer  hold  the  same  skills  for  their  whole  career  lives.  New  skills  and  new  knowledge  is  always 
needed  on  the  job  and  in  the  labor  market. 

One  of  the  biggest  challenges  is  the  low  awareness  of  the  importance  of  training  in  the  Chinese  government 
or  business  organizations.  There  is  currently  very  limited  public  fund  or  financial  aid  for  1)  previous  laid-off  workers 
to  acquire  new  skills  for  their  new  jobs  or  2)  on-job  training  for  employees  who  are  required  to  perform  new  skills. 
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Who  is  going  to  pay  for  training  and  learning  became  the  key  factor  that  hinders  the  development  of  Chinese  online 
work-based  training.  Chinese  Internet  users  usually  regard  the  resources  on  Internet  as  free  of  charge,  so  culturally 
nobody  is  willing  to  pay  the  bill  for  online  trainings. 

However,  the  fact  is  that  China’s  fast  and  stable  economic  growth  provides  a huge  incubating  bed  for 
web-based  training.  As  part  of  the  informal  learning  based  on  Internet,  it  is  very  important  for  Chinese  policy  makers 
and  service  provider  to  facilitate  this  web-based  learning  environment.  Also,  people’s  perspective  on  training  and 
lifelong  learning  needs  to  be  changed  in  order  to  match  the  fast  development  of  economics  and  the  increased  demand 
of  knowledge.  From  college  study  to  lifelong  learning  can  be  an  innovative  way  to  the  future  of  China. 

Final  thoughts 

Internet  has  opened  a new  gate  for  Chinese  education  with  opportunities  and  challenges.  There  are  more 
questions  needed  to  be  examined:  what  is  the  essential  requirement  for  Chinese  online  distance  learning?  What  is  the 
possible  global  partnership  for  Chinese  educational  technology  industry?  How  can  online  distance  learning  maximize 
the  educational  benefits  to  Chinese  society,  including  formal  and  informal  education,  in  both  school  and  business 
settings? 

The  author  believed  that  1)  Virtual  university  has  a high  potential  in  Chinese  education  market.  Chinese 
higher  institutions  have  great  opportunities  to  cooperate  with  foreign  universities  to  offer  better  and  reliable  online 
degree  education  2).  More  funding  and  pedagogical  research  need  to  be  provided  to  school-level  education  to  enhance 
Internet-based  instruction  and  study-help.  3)  In  order  to  offer  better  education  for  new  generation,  who  was  born  and 
raised  up  in  this  information  age,  Chinese  government  policy-makers  need  to  bridge  the  gap  between  higher  education 
and  life-long  learning  skill  set. 

Online  education  development  in  China  is  not  a linear  development  process,  but  a systematic  change 
influenced  by  economic  development,  and,  at  the  same  time,  relating  to  government  policies,  pedagogical  shifts,  and 
also  cultural  perspectives.  The  author  believes  that  despite  of  the  hardships,  Internet  and  its  web-based  application  has 
been  offering  Chinese  education  a high  way,  safe  way  and  innovative  way  to  the  future. 
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Abstract 

This  study  describes  the  communication  practices  of  the  Leadership  Team  (LT)  in  the  Metropolitan  School 
District  of  Decatur  Township  (MSDDT),  focusing  on  the  members’  values  and  beliefs  on  communication  within 
the  team,  communication  deficiencies  and  communication  channels  and  sources.  The  purpose  of  the  study  was 
to  improve  the  guidance  offered  by  the  School  System  Transformation  (SST)  protocol  to  the  MSDDT  by 
prescribing  preventive  measures  that  could  reduce  communication  deficiencies  in  their  LT. 

Introduction 

Why  systemic  change  is  important 

Over  the  20th  century,  various  school  reform  efforts  were  made  to  improve  existing  educational  systems  to 
better  serve  communities  (Schlechty,  1990).  However,  experience  has  indicated  that  school  reform  as  piecemeal 
change  is  not  enough  to  improve  the  overall  system,  whether  the  system  is  a classroom,  a school,  schools  within 
schools,  or  school  districts  (Reigeluth,  1999).  A systemic  change,  which  must  include  all  aspects  of  the  system, 
should  be  a true  transformation.  According  to  Reigeluth  and  Joseph  (2001),  a systemic  change  approach  in  K-12 
settings  could  bring  significant  improvements  to  the  educational  experience  of  students  and  their  families, 
school  employees  and  the  entire  community. 

Jenlink  et  al  (1998)  define  systemic  change  as  an  approach  that  recognizes  the  “interrelationships  and 
interdependencies  among  the  parts  of  the  educational  system,  with  the  consequence  that  desired  changes  in  one 
part  the  system  must  be  accompanied  by  changes  in  other  parts  that  are  necessary  to  support  those  desired 
changes”  (p.  219).  It  also  recognizes  the  “interrelationships  and  interdependencies  between  the  educational 
system  and  its  community,  including  parents,  employers,  social  service  agencies  and  much  more,  with  the 
consequences  that  all  those  stakeholders  are  given  active  ownership  over  the  change  effort”  (p.  219). 

Guidance  for  the  systemic  change  process 

In  today’s  US,  a few  school-  and  district-wide  systemic  change  efforts  have  been  implemented.  Several 
design  theories  were  developed  to  guide  change  efforts.  One  big  category  of  these  design  theories  is  guidance 
primarily  about  what  the  school  should  be  like,  not  the  systemic  change  process.  These  included  two  nationally 
recognized  educational  change  methods,  the  Coalition  of  Essential  Schools  (CES)  and  Success  for  All-Roots  & 
Wings  (SFA).  CES  (Sizer,  1984;  2002)  is  a system  that  promotes  ten  common  and  eight  organizational 
principles  as  the  foundation  of  their  philosophy  of  schooling  to  restructure  secondary  schools.  SFA  (Slavin  & 
Madden,  2001;  Stringfield,  Ross  & Smith,  1996)  is  a school-wide  change  model  with  an  underlying  principle 
that  promises  “every  child"  will  be  successful  in  learning  to  read  through  their  curriculum.  Other  design  theories 
about  the  change  products  are  the  School  Development  Program  (SDP),  and  Knowledge  Work  Supervision® 
(KWS).  SDP  (Comer,  Haynes,  Joyner  & Ben-Avie,  1996)  is  a model  that  advocates  on  behalf  of  parents  and 
families  to  have  a central  role  in  the  change  process  for  the  education  of  their  children.  KWS  (Duffy,  Rogerson, 
& Blick,  2000)  is  a methodology  that  assists  the  redesign  of  an  entire  school  district  from  an  innovative  point  of 
view  by  recognizing  three  fundamental  principles:  a.  understanding  how  systems  change,  b.  working  with 
individuals  as  well  as  with  groups,  and  c.  developing  necessary  attitudes  and  skills  of  those  facilitating  the 
change. 
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On  the  other  hand,  theories  guiding  school  change  process  include  GSTE  and  SUTE.  The  GSTE  (Jenlink, 
Reigeluth,  Carr,  & Nelson,  1996;  1998)  is  a guidance  model  to  facilitate  systemic  change  in  K-12  school 
districts.  Duffy  also  developed  a framework  to  guide  transformational  change  in  school  districts,  which  is  called 
Step-Up-To-Excellence  (SUTE).  GSTE  and  SUTE  were  blended  into  a new  hybrid  methodology,  which  is 
called  the  School  System  Transformation  (SST)  Protocol  (Reigeluth,  C.  & Duffy,  F,  2008).  It  has  some 
sequential  elements  and  some  elements  that  need  to  be  addressed  continuously  throughout  the  transformation 
process.  The  elements  fall  into  five  phases:  prepare,  envision,  transform,  sustain  and  evaluate.  Each  step  in  the 
protocol  is  not  in  a lock-step  sequence  and  should  be  perceived  as  a set  of  flowing  activities  that  converge, 
diverge  and  backflow  from  time  to  time.  Each  phase  has  several  steps,  and  each  step  has  multiple  tasks  and 
activities.  The  SST  Protocol  has  been  applied  as  guidance  for  the  change  effort  in  the  Metropolitan  School 
District  of  Decatur  Township  (MSDDT)  in  Indianapolis.  One  of  the  phases  in  the  SST  Protocol  addresses  the 
process  of  forming  and  participating  in  a Leadership  Team  (LT),  which  could  either  promote  the  success  or 
ensure  the  failure  of  a district-wide  change  effort,  and  thus  is  the  focus  of  this  study.  The  performance, 
contribution  and  impact  of  a LT  in  a system  could  be  strongly  influenced  by  the  efficiency  of  communication 
among  team  members. 

Communication  theories  related  to  systemic  change 

The  traditional  definition  of  communication  is  the  exchange  of  information,  ideas  and  feelings  in  order  to 
get  a message  across  (Mclntire.,  & Fessenden.  1994).  There  are  various  types  of  communication:  intrapersonal, 
interpersonal,  public,  mediated,  organizational,  intercultural  and  mass  (Fiordo,  1990).  Intrapersonal 
communication  takes  place  within  each  of  us  as  we  talk  to  ourselves;  interpersonal  communication  describes 
communication  between  participants  who  are  dependent  upon  one  another  and  have  a shared  history;  public 
communication  refers  to  the  sending  of  messages  to  an  audience;  mediated  communication  pertains  to  the  use  of 
various  technologies  to  mediate  the  sending  and  receiving  of  meaning  and  messages;  organizational 
communication  refers  to  messages  transacted  within,  from  and  to  an  organization;  intercultural  communication 
applies  to  sending  and  receiving  messages  between  organizations  and  individuals  from  different  cultures  and 
societies;  mass  communication  is  about  how  individuals  and  entities  relay  information  through  mass  media  to 
large  segments  of  the  population  at  the  same  time. 

Considering  the  above  categorization  of  communication,  given  the  purpose  of  this  study,  in  this  phase  of 
the  SST,  attention  will  only  be  given  to  interpersonal,  public,  mediated  and  intercultural  communications 
because  they  are  directly  related  to  communication  skills  the  team  members  are  expected  to  acquire, 
communication  channels  and  sources  desired  by  the  team  and  how  to  improve  communication  deficiencies 
among  different  stakeholder  groups.  In  the  later  phases  of  the  research,  organizational  communication  will  be 
touched  upon  since  the  findings  from  the  analysis  of  the  current  data  collected  strongly  suggest  a close 
relationship  between  the  internal  LT  communication  and  the  LT's  communication  with  other  stakeholder  groups 
outside  the  team.  Intrapersonal,  intercultural  and  mass  communication  will  be  skipped  in  this  study  because  it 
specifically  focuses  on  communication  within  the  leadership  team  under  analysis. 

Although  issues  related  to  leadership  communication  have  been  well  researched  in  both  school  and 
corporate  settings,  researchers  usually  tend  to  focus  on  communication  strategies  or  skills  leaders  or  executives 
are  supposed  to  have  in  order  to  guide,  direct,  motivate  and  inspire  people  through  effective  communication.  For 
example,  Deborah  Barrett  summarized  and  proposed  several  leadership  communication  frameworks,  arguing 
that  leadership  communication  consists  of  layered,  expanding  skills  from  core  strategy  development  and 
effective  writing  and  speaking  to  the  use  of  these  skills  in  more  complex  situations  (Barrett,  2008).  However, 
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some  of  the  frameworks  and  skills  might  not  apply  to  the  LT  of  MSDDT,  because  the  team  consists  of  different 
stakeholders  in  the  school  district,  including  administrators,  teachers,  parents,  community  members,  support 
staff,  school  board  and  ministerial  staff. 

The  composition  of  the  team  reflects  the  application  of  the  stakeholder  participation  concept.  Stakeholder 
participation  has  been  defined  as  "a  process  whereby  stakeholders  - those  with  rights  (and  therefore 
responsibilities)  and/or  interests  - play  an  active  role  in  decision-making  and  in  the  consequent  activities  which 
affect  them”  (SDD,  1995,  p.  5).  Stakeholder  participation  in  the  decision-making  of  educational  systems  can 
potentially  improve  their  design  and  implementation  by  improving  ownership,  building  consensus,  helping  to 
reach  disadvantaged  groups,  mobilizing  additional  resources,  and  building  institutional  capacity  (Colletta  & 
Perkins,  2007). 

Research  questions 

The  LT  in  MSDDT  was  formed  under  the  guidance  of  the  SST  protocol  to  fulfill  the  mission  of  guiding  the 
change  process  in  the  township.  This  kind  of  LT  is  significant  in  today’s  educational  setting.  An  industrial-age 
mindset  of  decision-making  has  prescribed  change  in  education  for  many  years  through  a top-down 
decision-making  structure  that  functions  with  little  input  from  its  stakeholders.  The  introduction  of  decision 
making  teams  (comprised  of  diverse  stakeholders  in  the  organization)  into  the  leadership  of  organizations  has 
revolutionized  the  way  these  organizations  function  and  produce  (Reigeluth  & Duffy,  2008). 

Senge  (1990)  argues  that  a learning  team  within  an  organization  has  the  capacity  to  think  together  and 
dialogue  with  the  purpose  of  learning  together.  He  states  that  team  learning  is  the  process  of  aligning  and 
developing  the  capacity  of  a team  to  create  the  results  its  members  truly  desire.  Duffy  and  colleagues  (2000) 
proposed  a systemic  change  process  to  transform  school  systems  into  high-performing  organizations  of  learners 
through  team-based  design  work.  Some  advantages  of  using  teams  in  systemic  change  efforts  include:  a)  teams 
increase  participation  and  collaboration,  which  could  increase  motivation,  job  satisfaction,  and  commitment,  b) 
teams  dissolve  hierarchies  within  institutions,  creating  opportunities  for  communication  and  collaboration,  c) 
teams  promote  conditions  for  creation  and  diffusion  of  knowledge,  and  d)  those  who  are  closest  to  the  work 
understand  best  how  to  improve  it  (Duffy  et  al.,  2000).  It  can  be  observed  that  communication  permeates  in  all 
these  factors  in  order  for  them  to  be  successfully  achieved.  In  addition  to  communication  skills  each  member  is 
required  to  possess,  this  study  seeks  to  investigate  whether  the  team  culture  and  climate  are  conducive  to 
communication  and  whether  communication  channels  and  sources  are  supportive  of  team  communication.  These 
issues  were  developed  into  research  questions  to  examine  the  current  status  of  internal  communication  within 
the  LT  of  MSDDT. 

The  research  questions  are: 

1 . What  are  the  implied  communication  patterns  of  the  LT  (who  do  members  share  ideas  with  and  how 
often)?  What  characteristics  of  communication  exist  within  the  team? 

2.  What  are  team  members’  values,  beliefs  and  perceptions  about  their  team  communication? 

3.  What  are  the  means  and  sources  of  communication  within  the  team? 

4.  What  are  the  communication  deficiencies  and  how  they  can  be  improved? 

Methods 

This  research  study  employed  a formative  research  methodology,  which  follows  a case  study 
methodological  approach  in  qualitative  research  (Reigeluth  & Frick,  1999,  p.  637).  According  to  Reigeluth  and 
Frick,  formative  research  asks  three  basic  questions:  1)  What  is  working?  2)  What  needs  to  be  improved?  And  3) 
how  can  it  be  improved?  Formative  research  is  a kind  of  developmental  research  or  action  research  that  is 
intended  to  improve  design  theory  for  designing  instructional  practices  or  processes  (Reigeluth  & Frick,  1999). 
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This  methodology  is  useful  in  identifying  what  worked  for  communication  at  Leadership  Team  (LT)  meetings  in 
MSDDT  and  how  communication  could  have  been  improved,  indicating  possible  additions  to  the  SST  Protocol 
to  improve  communication  at  LT  meetings. 

Formative  research  classifies  case  studies  as  either  designed  or  naturalistic  cases.  According  to  Reigeluth 
and  Frick  (1999),  formative  research  is  a designed  case  when  a theory /model  is  instantiated  and  then  formatively 
evaluated.  In  contrast,  formative  research  is  a naturalistic  case  when  the  case  selected  was  not  specifically 
designed  according  to  the  theory  but  serves  the  same  goals  and  contexts  (Pascoe,  2008).  According  to  Reigeluth 
and  Frick  (1999),  in  naturalistic  cases  the  formative  evaluation  of  the  instantiation  is  done  during  its  application. 
Therefore,  this  research  study  was  a naturalistic  case,  primarily  because  the  theory  did  not  offer  guidelines  to 
enhance  communication  at  LT  meetings,  and  it  was  studied  post  facto,  whereby  formative  evaluation  occurred 
after  the  case  had  already  taken  place.  A naturalistic  study  is  usually  conducted  based  on  the  following 
procedure:  1.  Select  an  existing  theory;  2.  Select  a case;  3.  Collect  and  analyze  formative  data  on  the  case;  4. 
Offer  tentative  revisions  for  the  theory  (Reigeluth&  Frick,  1999).  With  this  procedure  serving  as  the  basic 
framework  of  the  current  study,  the  researcher  aims  at  revising  the  current  SST  theory  relevant  to  the 
communication  of  the  LT. 

School  district 

The  school  district  selected  for  this  study  was  the  Metropolitan  School  District  of  Decatur  Township 
(MSDDT)  in  the  State  of  Indiana,  US,  with  an  approximate  population  of  24,000.  It  was  one  of  eleven  public 
school  corporations  in  Indianapolis  Marion  County,  and  at  the  time  of  the  study  served  approximately  5,500 
students.  MSDDT  offered  one  centralized  early  childhood  program,  four  elementary  schools  (grades  1-4),  two 
intermediate  schools  (5-6),  one  middle  school  (7-8),  and  one  central  high  school  having  five  academies  (9-12). 
At  the  time,  students  were  served  by  almost  260  full-time  teachers  and  more  than  280  professional  staff 
members.  Approximately  90  percent  of  MSDDT  students  were  considered  white,  and  some  40  percent  of  the 
student  population  received  free  or  reduced  lunches  and  textbooks  (Pascoe,  2008).  In  2001,  MSDDT  and 
Indiana  University  established  a partnership  to  enhance  the  educational  opportunities  offered  to  students,  their 
families,  and  the  community-at-large  through  a district-wide  systemic  change  process  using  the  SST. 

Leadership  Team 

In  the  spring  of  2001,  a predecessor  of  the  Leadership  Team,  named  the  Core  Team,  was  formed.  They  met 
as  a team  to  identify  core  values  and  ideas  that  could  guide  the  MSDDT  to  improve  its  educational  process  and 
include  all  stakeholders  in  reaching  consensus  on  the  changes  that  could  most  benefit  their  students.  This  Core 
Team  met  almost  every  week  until  the  end  of  the  fall  of  2002,  at  which  point  they  expanded  into  the  LT,  though 
they  continued  to  meet  as  a Facilitation  Team  for  planning  the  LT  meetings.  In  February  2003  the  newly  formed 
LT  had  a broad  representation  of  MSDDT  stakeholder  leaders,  including  community  members,  to  work  together 
to  design  better  learning  experiences  for  students  in  this  school  district. 

From  mid-November  2003  through  April  2004  a second  phase  of  the  LT  took  place.  The  LT  was 
reconstituted  and  expanded,  and  devoted  time  to  reviewing  data  gathered  from  stakeholders  in  the  school  system 
about  the  need  for  systemic  change  in  their  school  district.  Next,  the  LT  developed  a Framework  of  mission, 
vision,  and  ideal  beliefs  about  education,  developed  in  collaboration  with  the  community,  which  became 
prominent  throughout  the  MSDDT.  The  current  LT  has  forty  members,  including  eleven  administrators,  eleven 
teachers,  seven  parents,  four  community  members,  two  support  staff,  one  school  board  member,  one  ministerial 
member  and  three  facilitators. 
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Data  Collection  and  Analysis 

In  the  study,  observation,  survey,  focus  group  and  individual  interviews,  and  documents  were  used  as  main 
techniques  for  data  collection.  Hand  analysis  was  used  as  the  analysis  approach  and  findings  were  validated 
through  triangulation  and  member  checking. 

Observation 

Assuming  a role  of  nonparticipant  observer,  the  researcher  observed  LT  meetings,  which  were  held  once  a 
month.  The  researcher  attended  meetings  and  observed  discussions  to  further  inform  the  development  of  the 
protocol.  Field  notes  were  produced  during  three  observation  sessions  in  November  and  December  of  2008,  with 
each  lasting  2 hours,  about  communication  and  types  of  information  considered  and  used.  The  field  notes  were 
used  for  identifying  themes  related  to  communication  in  the  LT  and  for  informing  the  survey  questions  and 
interview  protocols  later  developed. 

Survey 

A short  survey  questionnaire  with  three  questions  was  sent  out  to  the  entire  LT  to  capture  the  characteristics 
of  the  team's  communication  and  the  members’  perspectives  on  good  communication  and  leadership  skills.  In 
order  to  get  the  most  responses  from  the  team  members,  the  survey  was  made  as  short  as  possible.  It  was  sent 
out  via  email  together  with  a participant  information  sheet,  and  participants  were  reminded  to  reply  only  to  the 
researcher  to  protect  their  confidentiality.  The  survey  was  sent  out  multiple  times,  and  twelve  out  of  thirty  seven 
members  (excluding  the  facilitators)  filled  out  the  survey.  The  twelve  respondents  include  five  administrators, 
four  teachers  and  three  community  members.  The  sample  has  limited  representativeness  of  the  whole  LT,  but  the 
majority  stakeholders  of  the  LT  were  included  in  the  sample.  The  information  about  which  stakeholder  group 
each  person  was  in  is  known.  The  three  questions  asked  were: 

1.  Within  the  LT,  name  the  people  with  whom  you  are  most  likely  to  share  new  ideas.  What  is  the  frequency 

you  communicate  with  each  other  related  to  the  LT  issues?  (the  number  of  times  per  day  or  per  week  or  per 

month) 

2.  Within  the  LT,  what  people  would  you  be  most  likely  to  choose  as  the  team  leader  if  you  were  to  work  on  a 

team  project?  And  why? 

3.  If  you  were  going  on  a vacation,  name  the  members  on  the  Leadership  Team  you  would  like  to  go  with. 

Why  them? 

Interviews 

Eight  members  of  the  team  were  interviewed.  Three  of  them  were  teachers,  three  of  them  were  community 
members,  one  person  was  on  the  school  board  and  one  was  a school  principal.  It  is  noticeable  that  this  sample  is 
not  sufficiently  representative  of  the  whole  leadership  team.  The  generalizability  of  the  study  results  is  thus 
reduced.  The  years  they  had  been  on  the  team  ranged  from  one  to  seven  years.  The  community  members 
included  parents,  retired  teachers  and  people  who  were  from  a university  located  in  the  same  city  as  the 
township.  All  of  them  had  been  living  in  the  township  for  more  than  fifteen  years.  Two  focus  groups  were 
conducted  with  three  teachers  and  two  community  members  based  on  a semi-structured  interview  protocol.  Each 
focus  group  lasted  about  30  minutes. 

Three  one-on-one  interviews  were  conducted  with  a principal,  a community  member  and  a member  from 
the  school  board,  with  each  lasting  about  40  minutes.  Semi-structured  interview  protocols  were  used,  and 
open-ended  questions  were  asked  to  allow  maximal  flexibility  for  participants’  responses.  The  researcher  took 
brief  notes  during  the  interviews,  and  all  the  interviews  were  audio  taped  and  transcribed. 

Documents 

Public  documents,  such  as  minutes  from  LT  meetings,  official  memos,  newsletters,  notes,  records,  and 
physical  and  on-line  archival  materials  were  collected  as  sources  to  make  sense  of  communication  channels  and 
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patterns  in  the  group.  Notes  were  taken  about  the  documents  to  record  information  from  them.  The  documents 
were  used  to  gain  a sense  of  the  context  and  language  used  and  to  trace  the  linkages  of  discourse  related  to  key 
themes. 

Data  analysis 

All  the  notes  and  materials  obtained  were  organized  by  type,  participant,  site  and  combinations  of  these 
categories.  Duplicate  copies  of  all  forms  of  data  were  kept.  All  the  interviews  and  observational  notes  were 
transcribed.  The  data  analysis  was  conducted  following  data  analysis  activities  of  data  reduction,  data  display, 
and  conclusion  drawing  (Miles  & Huberman,  1984).  Data  reduction  is  “selecting,  focusing,  simplifying, 
abstracting,  and  transforming  the  ‘raw'  data....”  (Miles  & Huberman,  1984,  p.  21).  Summary  information  could 
be  placed  in  a series  of  matrices  which  specify  relevant  situational  characteristics  and  array  categories  of  data 
(Miles  & Huberman,  1984).  Guided  by  this  process,  the  researcher  read  the  data,  marked  them  by  hand,  divided 
them  into  parts  and  created  a database.  To  narrow  the  text  data  into  a few  themes,  the  coding  process  followed 
was  to  make  sense  of  the  whole  data,  divide  them  into  text,  label  the  segments  with  codes,  examine  codes  for 
overlap  and  redundancy,  and  collapse  these  codes  into  broad  themes.  The  researcher  scanned  theory  and 
methods  literature  for  ideas  that  would  help  make  sense  of  emerging  themes  and  concepts.  Analytic  memos  and 
theoretical  notes  with  commentary  about  emerging  themes,  anomalies  of  inconsistencies  and  relationships  were 
created.  Portrayals  of  each  theme  were  written,  and  tables  and  charts  were  produced  to  represent  the  findings. 

Validating  findings 

The  findings  were  validated  through  the  strategies  of  triangulation  and  member  checking.  As  stated  in  the 
data  collection  section,  the  researcher  triangulated  among  different  data  sources  to  enhance  the  accuracy  of  the 
study.  With  different  types  of  data  (observational  field  notes,  interviews,  notes  on  documents),  and  different 
methods  of  data  collection  (observation,  interview,  document  analysis),  the  information  was  drawn  from 
multiple  sources,  which  enhanced  the  accuracy  and  credibility  of  the  findings.  The  researcher  also  checked  notes 
and  transcriptions  with  participants  in  the  study  to  determine  and  enhance  credibility. 

The  findings  will  be  checked  back  to  participants,  and  they  will  be  asked  via  email  about  the  accuracy  of 
the  report,  whether  the  description  is  complete  and  realistic,  and  if  the  interpretations  are  fair  and  representative. 
The  report  will  be  sent  to  them  when  the  study  is  finished. 

Findings 

The  findings  are  organized  in  relation  to  the  research  questions.  They  start  with  the  characteristics  of  the 
team’s  communication,  communication  skills  admired  by  the  team  members,  followed  by  the  team  members’ 
values,  beliefs  and  perceptions  about  their  team  communication,  the  means  and  sources  of  communication 
within  the  team  and  the  communication  deficiencies  detected  by  the  members. 

Team  communication  characteristics 

The  researcher  intended  to  capture  the  characteristics  of  the  team’s  communication  by  asking  questions 
examining  who  the  members  were  that  most  team  members  would  like  to  share  ideas  with.  After  analyzing  the 
answers  given  by  twelve  members,  five  people  from  the  administrator  group  and  three  teachers  from  the  teacher 
group  were  most  frequently  chosen  as  people  to  share  ideas  with.  It  also  implies  that  people  intended  to  choose 
people  from  the  same  stakeholder  groups  as  themselves.  Although  the  result  was  not  based  on  the  responses 
from  all  the  team  members,  it  appears  that  communication  based  on  hierarchical  ranks  still  plays  a role  in  the 
team.  However,  another  explanation  is  that  since  the  LT  only  met  monthly,  the  administrators  were  the  team 
members  who  met  most  often,  or  administrators  were  in  charge  of  the  meeting  planning  and  management, 
including  member  recruiting,  therefore  they  were  more  familiar  with  most  of  the  members.  It  needs  to  be  noted 
that  none  of  the  community  members,  parents,  support  staff  or  school  board  members  were  chosen,  which 
means  there  must  have  been  some  communication  breakdowns  between  teacher  and  administrator  groups  and 
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the  members  outside  the  school.  Another  explanation  to  this  phenomenon  could  be  that  a group  of  the  most 
influential  people  has  emerged  in  the  team.  These  discoveries  shed  light  on  developing  the  interview  protocol  at 
the  later  stage  of  the  study  and  the  reasons  underneath  were  explained  in  the  later  section. 

For  all  the  respondents  who  did  reply  that  there  were  certain  people  with  whom  they  would  like  to  talk 
about  LT  issues,  most  of  them  only  met  each  other  once  or  twice  a month,  with  only  a few  exceptions  for  some 
teachers,  but  most  teachers  were  in  different  schools,  so  they  could  not  see  each  other  very  often.  It  appeared 
that,  except  for  the  monthly  LT  meetings,  there  was  not  enough  face-to-face  communication  for  team  members, 
even  with  the  ones  with  whom  they  would  most  like  to  share  ideas. 

According  to  the  answers  respondents  gave  to  the  survey,  it  was  found  that,  other  than  formal  relationships 
developed  based  on  working  relations  between  the  team  members,  informal  relationships  did  emerge  within  the 
group.  Nine  out  of  twelve  respondents  chose  some  team  members  as  their  vacation  partners.  However,  3 of  them 
refused  to  choose  anyone  as  a vacation  partner  because  they  didn't  know  anyone  well  enough.  The  reasons 
people  chose  others  as  vacation  partners  included:  same  hobbies  and  interests,  sense  of  humor,  same  age, 
fascinating  personalities,  know  them  longest.  Furthermore,  the  partners  chosen  were  always  within  their  own 
stakeholder  groups,  which  means  teachers  tended  to  choose  teachers  and  administrators  tended  to  choose 
administrators,  and  people  working  in  school  never  choose  community  members. 

Leadership  and  communication  skills  admired 

According  to  the  answers  to  the  survey,  four  people  from  the  administration  group  and  two  from  the 
teacher  group  were  chosen  as  team  leaders,  for  they  possessed  certain  leadership  and  communication  skills 
desired  by  other  members.  These  skills  are  summarized  below:  stays  on  top  of  the  issues;  would  do  their  share 
and  designate  out  some  responsibilities;  work  in  a professional  and  respectful  manner;  will  not  sugar  coat 
information;  accept  the  ideas  of  others;  understand  my  job;  thorough;  efficient;  creative;  hard  working;  high 
work  ethic;  have  access  to  many  resources;  passion  for  students  and  learning;  make  a person  feel  needed  and 
appreciated;  organization  skills;  knowledge  of  district;  not  afraid  to  do  work  themselves;  out  of  the  box  thinking; 
keep  the  team  focused;  not  hesitant  to  speak;  not  beat  around  the  bush. 

Team  members’  values,  beliefs  and  perceptions  on  team  communication 

Interviewees  were  asked  about  the  leadership  role  and  the  team  vision,  mission  and  products  to  examine  if 
this  information  had  been  effectively  communicated  to  the  members.  Most  of  the  interviewees  agreed  that  the 
LT  provided  them  with  a framework  and  a bigger  picture  of  where  the  township  needed  to  go.  It  helped  them 
look  at  a more  global  picture  of  all  the  stakeholders  involved  in  the  reinvention  process  and  come  to  know 
different  perspectives  from  stakeholders.  Although  not  every  member  was  clear  about  the  vision,  mission  and 
products  of  the  LT,  the  members  who  provided  answers  agreed  that  the  mission  of  the  LT  was  to  assist  the  rest  of 
the  school  system  with  guidelines  to  improve  schools.  In  addition,  they  were  also  aware  that  the  mission  of  the 
LT  might  change  over  time. 

Compared  with  the  vision  and  mission  of  the  LT,  the  members’  opinions  on  its  products  varied  from  person 
to  person.  The  vision  and  mission  statement  was  considered  a major  product  of  the  team.  Some  considered  that 
the  major  product  was  sharing  ideas  and  getting  feedback  from  different  stakeholders,  and  some  believed  the 
products  of  the  leadership  team  were  the  tools  or  guidelines  that  support  the  schools  to  form  their  teams  and 
drive  their  reinvention  processes.  The  inability  to  state  the  vision  and  mission  of  the  LT  and  the  mixed 
understanding  of  its  products  reflects  that  these  concepts  were  not  effectively  communicated  to  the  members, 
which  was  again  confirmed  by  the  answers  given  by  some  interviewees.  Two  members  emphasized  that  the 
vision  and  mission  of  the  LT  was  really  cloudy  at  one  point. 

However,  both  of  them  also  pointed  out  that  the  goal  of  the  team  had  been  defined  in  recent  months  and  the 
team  had  been  back  on  the  right  track.  That  largely  was  due  to  stronger  leadership  from  the  central  office,  which 
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worked  hard  to  reenergize,  refocus  and  redirect  the  team.  Other  members  expressed  the  opinion  that  the  LT  had 
had  thorough  conversations  and  follow-up  concerning  its  vision,  mission  and  products.  The  seeming 
ineffectiveness  might  have  been  caused  by  some  individuals  not  taking  an  active  role  or  by  gaps  due  to  absences 
from  the  meetings. 

In  order  to  examine  if  the  team  had  effectively  communicated  among  its  members  about  their  particular 
roles  in  the  team,  participants  were  asked  about  this.  Almost  every  member  interviewed  was  clear  about  their 
particular  role  in  the  team,  at  least  conceptually,  since  some  community  members  were  not  very  positive  as  to 
whether  they  had  met  the  requirements  of  their  roles.  Generally  speaking,  the  participants  thought  that  they  were 
expected  to  actively  participate  in  the  meetings,  bring  in  their  ideas  and  thoughts  and  share  their  point  of  view 
with  each  other.  They  were  specifically  clear  about  their  bridging  role  to  communicate  between  the  LT  and  their 
own  stakeholder  group,  although  this  clarification  was  reinforced  only  six  months  earlier,  which  is  consistent 
with  the  time  talking  points  were  sent  out  through  email  to  each  member.  This  demonstrates  that  the  team  had 
been  making  efforts  to  emphasize  the  importance  of  communicating  with  all  stakeholder  groups. 

It  is  worth  noting  that  the  teachers  interviewed  universally  considered  themselves  as  an  integral  part  of  the 
reinvention  process  and  had  high  demands  on  themselves  in  terms  of  changing  mental  models,  communicating 
within  the  team,  learning  related  knowledge,  and  being  supportive  in  their  own  buildings  to  move  the 
reinvention  process  forward.  Some  mentioned  that  they  attended  administration  courses  or  went  to  retreats.  By 
comparison,  although  community  members  were  clear  that  each  was  expected  to  bring  an  outsider  perspective, 
they  stated  their  concerns  that  sometimes  they  felt  lost  and  didn't  really  know  whether  they  were  making 
contributions  to  the  team.  The  researcher  further  explored  this  issue  and  examined  the  reasons  by  asking 
questions  related  to  these  communication  deficiencies. 

Sources  and  means  of  communication 

For  the  sources  and  channels  of  communication  within  the  leadership  team,  all  the  members  interviewed 
shared  the  opinion  that  key  people  on  the  team,  especially  those  on  the  Worker  Bee  team,  served  as 
communication  sources.  In  addition,  a new  communication  strategy,  which  had  only  been  adopted  six  months 
earlier,  had  proved  very  effective.  That  is  the  central  office  staff  summarized  the  “talking  points”  from  each 
meeting  and  sent  them  to  all  members  via  email.  According  to  them,  the  talking  points  had  been  useful  because 
they  served  as  a good  reminder  of  what  they  did  in  each  meeting  and  provided  a good  way  for  them  to  share 
what  they  had  done  in  the  LT  with  their  stakeholder  groups.  And  to  a large  extent,  those  talking  points  were 
produced  particularly  for  the  members  to  take  back  to  their  own  stakeholder  group  members. 

It  needs  to  be  noted  that,  although  every  participant  was  notified  before  the  interview  that  the  study  was  to 
examine  the  internal  communication  within  the  LT,  almost  every  participant  touched  on  the  issues  of 
communication  between  the  team  and  other  stakeholder  groups  and  provided  suggestions.  That  feedback, 
although  not  the  initial  focus  of  the  study,  was  good  to  have  in  that  the  issue  of  communication  with  other 
stakeholder  groups  was  consistent  with  the  participants'  understanding  of  the  LT  role  and  the  composition  of  the 
team  itself. 

Communication  deficiencies 

The  lack  of  informal  communication  was  mentioned  by  every  member  interviewed  as  one  of  the 
communication  deficiencies  on  the  team.  It  is  consistent  with  the  survey  results  presented  above.  As  introduced 
by  the  members  above,  the  LT  meeting  itself  served  as  the  primary  source  of  communication.  However,  the 
meeting  was  only  held  once  a month,  therefore  most  members  interviewed  felt  there  was  a lack  of  understanding 
of  people  on  a personal  basis.  Some  of  them  didn’t  even  know  who  the  team  members  were  and  what  they  did. 
This  issue  is  more  obvious  with  community  members,  because  most  of  them  are  not  directly  involved  in  school 
and  have  fewer  opportunities  to  meet  with  other  community  members. 
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The  members  felt  that  there  had  been  too  many  mental  activities  and  idea  exchanging  taking  place,  which 
had  been  effective  to  learn  knowledge  but  ineffective  to  get  to  know  the  members'  needs.  Though  members 
were  divided  into  small  groups  to  work  on  certain  activities  almost  every  meeting,  they  didn’t  get  the 
opportunity  to  work  with  every  member  and  get  to  know  them.  The  lack  of  informal  communication  and  less 
understanding  of  other  team  members  contributed  to  a team  culture  that  is  more  alien  and  intimidating  for 
community  members  to  the  extent  that  several  of  them  started  to  question  their  function  and  contribution  to  the 
team. 

It  has  to  be  highlighted  that  all  the  community  members  interviewed  felt  uncomfortable  offering  their 
ideas  and  questioned  their  own  value  on  the  team,  and  they  also  analyzed  various  reasons  behind  that.  In 
addition  to  the  lack  of  informal  interaction  among  team  members  and  individuals'  personality  traits,  they 
deemed  the  structure  of  the  team  (25  educators  and  four  or  five  outsiders)  and  the  jargon  used  by  the  team  were 
primary  impediments  to  communication  between  members  of  the  LT.  They  thought  the  setting  of  mostly 
educators  in  the  group  not  only  was  intimidating  for  people  who  were  not  so  used  to  talking  in  front  of  big 
groups,  but  also  got  them  stuck  in  a mode  that  they  were  just  sitting  there  as  advisory  members  watching 
internal  people  really  doing  the  work. 

Management  ofLT  meetings 

All  the  participants  provided  positive  feedback  about  the  management  of  the  LT  meetings.  The  time  of  the 
meetings  worked  for  everybody’s  schedule.  Though  the  meetings  were  held  monthly,  most  interviewees  thought 
this  frequency  was  enough  because  they  understood  that  things  needed  time  to  develop.  They  also  praised  the 
organization  of  the  meetings  and  activities.  However,  people  also  acknowledged  that,  since  too  many  things 
were  going  on  in  each  meeting,  quite  often  they  felt  like  they  didn’t  quite  understand  the  goals  of  each  meeting 
and  got  nowhere  in  some  of  those  meetings.  One  common  opinion  was  that  the  agenda  of  each  meeting  was  too 
packed. 

The  participants  acknowledged  the  importance  of  the  activities  designed  by  the  Worker  Bee  team  for  each 
meeting,  and  some  of  them  were  deemed  very  effective.  For  instance,  several  of  the  interviewees  mentioned  the 
activity  in  which  they  shared  the  middle  school  redesign  process.  However,  there  were  complaints  that 
sometimes  it  seemed  they  were  doing  an  activity  that  was  well  planned  by  the  Worker  Bees  but  was  not  very 
relevant  to  the  real  world  or  its  purpose  was  not  so  clear,  and  they  did  not  feel  they  could  play  an  active  role  in 
it. 

The  participants  also  expressed  mixed  opinions  on  the  leadership  of  the  LT.  On  the  one  hand,  some  had 
seen  the  meetings  as  teacher-directed  because  the  teachers  usually  facilitated  and  directed  the  activities.  On  the 
other  hand,  some  stated  the  concern  that  they  felt  more  like  followers  than  leaders  in  the  group  because  often  the 
Worker  Bee  group  planned  all  the  activities.  This  indicates  that  the  LT  has  not  encouraged  the  members  to 
have  input  in  planning  the  meetings  or  activities.  Though  time  might  be  an  issue  with  most  of  the  members, 
being  provided  with  some  degree  of  decision  making  in  the  activities  they  will  do  in  the  meeting  was  not 
considered  a bad  thing. 

Discussion 

The  descriptions  from  the  interviewees  reflect  communication  issues  existing  both  within  the  team  and 
between  the  team  and  other  stakeholder  groups.  One  the  whole,  the  suggestions  provided  are  helpful  to  address 
several  of  the  eighteen  continuous  processes  in  the  SST  Protocol,  namely  sustain  motivation,  develop  and 
sustain  appropriate  leadership,  develop  group-process  and  team-building  skills,  build  team  spirit,  engage  in 
reflection,  communicate  with  stakeholders,  and  foster  organizational  learning.  These  suggestions  will  be 
discussed  in  relation  to  the  research  questions  explored  in  the  study  and  the  implications  for  the  SST  Protocol. 
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Improve  communication  deficiencies  & communicate  with  stakeholders 

One  member  suggested  that  assigning  stakeholder  jobs  in  the  team  might  be  a solution  to  bring  in  more 
parents  or  community  members,  which  is  also  a good  way  to  increase  opportunities  for  meeting  other  members 
more  informally.  Another  member  suggested  that,  in  order  for  people  to  have  a better  idea  of  how  each  member 
is  communicating  back  to  their  stakeholder  group,  the  LT  should  allocate  special  time  for  discussing  this  issue 
and  make  it  transparent  rather  than  continuing  to  keep  it  secretive.  To  help  people  know  each  other  much  better, 
they  expressed  a strong  hope  to  allocate  special  time  to  dealing  with  members’  questions  and  feedback, 
providing  clarification  if  it’s  needed.  A list  with  updated  member  information  was  highly  welcomed.  Moreover, 
the  members  expressed  their  willingness  to  see  more  mixed  or  shifted  groups  so  that  they  would  have  the  chance 
to  meet  and  work  with  more  members  on  the  team.  This  is  consistent  with  communicating  with  stakeholders, 
which  is  specified  in  the  SST  Protocol.  More  representatives  of  stakeholder  groups  in  the  team  will  bring  more 
perspectives  into  team  discussion  and  activities  and  facilitate  the  communication  between  the  school  and 
community.  Since  community  participation  is  so  important  in  the  whole  change  process,  it  is  suggested  that 
stakeholder  participation  should  be  emphasized  in  each  phase  of  the  five  phases  specified  in  the  SST  Protocol. 
Improve  communication  deficiency  & sustain  motivation  of  community  members 
Sustaining  motivation  is  an  important  process  out  of  the  18  processes  listed  in  the  SST  Protocol.  To 
encourage  community  members  to  contribute  to  the  team,  the  following  solutions  were  suggested: 

1.  Be  provided  with  a clearer  understanding  of  the  function  of  community  members  on  the  team  and 
historically  what  contribution  community  members  made  to  the  team. 

2.  Visit  schools  and  classrooms  in  order  to  have  a better  idea  about  what  is  going  on  in  the  school. 

3.  Recruit  student  members  on  the  LT  to  provide  more  concrete  information  to  the  community  members. 

4.  Rather  than  sitting  in  a large  group  meeting,  one-on-one  or  two-on-one  meetings  in  informal  surroundings 
(such  as  a coffee  shop)  might  be  more  comfortable  for  most  parents  and  community  members. 

5.  The  mentality  needs  to  be  changed  from  expecting  people  to  come  to  school  to  going  out  and  talking  to 
families. 

Improve  communication  channels  and  sources  & develop  group-process 

The  participants  identified  various  communication  channels  and  sources  that  they  thought  might  help 
address  communication  deficiencies  within  the  team.  These  suggestions  are  thought  to  improve  group-process 
developing  in  the  leadership  team  with  better  record  of  group  activities.  For  the  problem  of  not  remembering 
team  members'  names,  one  member  suggested  that  the  team  keep  a case  of  people’s  name  tags  so  that  each  one 
can  put  it  on  during  the  meeting  and  return  it  afterwards.  Another  suggestion  was  to  give  each  member  a binder 
or  a notebook  and  provide  updated  information  about  group  members.  Through  the  interviews,  the  researcher 
learned  that  there  had  been  a binder  for  each  member  several  years  ago  to  keep  related  readings  and  information 
sheets.  However,  the  binder  got  too  heavy,  and  people  just  ceased  bringing  them  to  the  meetings.  An  effective 
solution  might  be  keeping  the  binder  with  limited  pages,  but  no  matter  how  the  information  would  be  stored;  the 
members  need  to  receive  updated  information  on  each  member. 

To  facilitate  communication  both  within  the  LT  and  with  the  outside  stakeholder  groups,  it  is  suggested 
that  a website  could  be  built  holding  podcasting  and  vodcasting  of  each  LT  meeting  so  that  not  only  the  team 
members,  but  everyone  in  the  township  could  be  able  to  know  what  is  going  on  in  those  meetings.  This  would 
be  a good  way  to  spread  the  vision  of  the  LT,  keeping  the  teachers  and  community  informed  and  would  even  act 
as  a good  way  to  introduce  team  members. 

LT  meeting  management  & foster  team  learning 

Team  learning  within  the  LT  has  tremendous  significance  for  the  whole  district,  given  the  steering  role  it 
assumes.  Several  suggestions  were  made  on  the  management  of  the  meetings  in  order  to  foster  team  learning 
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and  for  all  the  members  to  be  clear  about  not  only  what  they  had  done  and  what  they  needed  to  do  in  each 
meeting,  but  more  importantly,  what  they  had  gained  from  what  they  had  done  and  what  they  still  felt  fuzzy 
about  and  needed  more  clarification  on. 

1.  Use  dinner  time  to  have  dinner  conversation,  reviewing  what  they  did  last  time  and  the  agenda  of  the 
meeting  on  that  day. 

2.  Pair  the  members  with  different  people  each  time  during  dinner  time. 

3.  Allocate  reflection  time  every  meeting  to  give  anyone  who  didn’t  speak  the  chance  to  speak  up. 

4.  Assign  one  person  in  charge  of  the  meeting  each  month.  Let  the  members  know  who  the  go-to  person  will 
be  if  they  have  problems  or  questions.  More  consistency  would  be  added  to  the  meetings  in  this  way. 

5.  It  may  take  more  than  a meeting  every  month  to  make  community  members  feel  involved  in  the  team. 
Therefore,  aside  from  the  LT  meetings,  three  or  four  key  members  of  the  LT  should  go  out  to  meet  with 
community  members  and  build  stronger  relationship  with  them. 

6.  If  the  LT  decides  the  meeting  frequency  is  not  enough  for  the  team,  it  is  suggested  to  make  each  meeting 
longer  rather  than  having  another  one  each  month. 

To  give  the  participants  a clearer  sense  of  those  meeting  activities,  one  implication  the  researcher  got  from 
the  interviewees  is  that  after  each  activity,  the  team  should  get  members'  feedback  or  at  least  provide  them  with 
an  evaluation  form  and  then  decide  whether  to  keep  the  activity  or  not.  A successful  activity  would  be  the  one 
that  is  relevant  to  educational  issues,  such  as  having  discussions  about  how  the  high  school  work  has  informed 
the  middle  school  work. 

One  of  the  implications  for  team  member  recruiting  is  that  the  potential  member  should  be  someone  who  is 
proactive,  aware  of  their  role  and  driven  to  assume  it,  comfortable  with  large  group  discussion  and  willing  to  be 
integrated  into  the  bigger  group,  and  motivated  to  learn  about  different  fields  of  knowledge.  It’s  better  if  they 
have  kids  or  someone  who  is  involved  in  the  school  system. 

Conclusion 

Communication  is  one  of  the  most  important  factors  that  determine  participants’  contribution  in  any  team 
effort,  particularly  an  effort  as  complex  as  a district-wide  systemic  change  process.  The  communication 
breakdowns  that  LT  members  experience  could  potentially  deny  the  success  of  the  effort.  This  research  study 
addressed  what  those  communication  deficiencies  are  and  how  to  improve  them.  When  members  of  a LT  are  not 
effectively  communicating,  opportunities  to  contribute,  grow  as  a group,  and  continue  the  progress  that  this 
particular  team  is  providing  to  the  system-wide  change  are  put  at  risk.  Hence,  studying  and  improving  the 
communication  of  LT  members  is  important  to  the  successful  implementation  of  a change  effort  in  a school 
district. 

For  the  eight  participants  who  did  accept  the  interview,  four  of  them  were  community  members,  whose 
perspectives  on  the  communication  problems  in  the  LT  are  definitely  different  from  members  of  other 
stakeholder  groups.  Therefore,  more  information  needs  to  be  collected  from  other  stakeholder  groups  to  verify 
the  interpretation  the  researcher  has  so  far. 

The  study  also  discovered  some  related  research  topics,  such  as  how  communication  affects  team  members' 
performance,  how  the  LT  should  improve  its  management  for  better  communication  to  take  place,  and  who 
should  assume  the  leadership  role  within  the  LT.  Therefore,  this  study  should  be  considered  as  a stepping  stone 
in  a long  series  of  research  studies  to  inform  LTs  and  school  districts  about  team  membership  interactions  and 
communication.  More  research  is  needed. 

Limitations  of  the  study 

The  limited  number  of  respondents  to  the  survey  and  limited  number  of  members  who  accepted  being 
interviewed  greatly  impact  the  credibility  of  the  study  results.  Unwillingness  to  participate  in  the  study  was 
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partly  due  to  bad  timing  (the  survey  and  interview  were  arranged  between  late  November  and  early  December, 
when  most  of  the  administrators  and  teachers  were  especially  busy  with  their  school  work)  and  perhaps  partly 
due  to  team  members  being  reluctant  to  talk  about  or  even  to  address  communication  issues  they  were  having. 
Another  possible  reason  is  that  they  were  just  not  interested  in  the  research  study  itself.  To  make  the  study 
complete  and  more  valid,  the  researcher  planned  a second  phase  of  data  collection,  and  tried  to  arrange  more 
interviews  with  the  teachers  and  administrators,  in  order  to  depict  a more  thorough  picture  of  communication 
within  the  Leadership  Team.  However,  the  second  phase  of  interview  did  not  materialize  due  to  the  LT  members' 
reluctance  to  accept  interviews.  A legitimate  explanation  for  this  could  be  that  the  LT  members  were  too  busy 
with  their  current  professional  roles  to  consider  volunteering.  Another  explanation  could  be  that  the  other  LT 
members  were  not  so  aware  of  communication  issues.  It  could  be  further  explained  that  other  team  members, 
different  than  some  community  members  and  some  teachers  who  accepted  most  interviews  of  the  study,  did  not 
consider  communication  was  a big  issue  for  the  team.  Hopefully  more  data  could  be  gathered  in  the  future  to 
enrich  the  findings  from  the  current  study. 
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Abstract:  We  report  on  the  design,  development,  implementation,  and  evaluation  of  a 
cased-based  instructional  environment  designed  for  learning  network  engineering  skills 
for  cybersecurity.  We  describe  the  process  of  design  and  development  research,  the 
societal  problem  addressed,  the  theory-based  solution,  and  the  evaluation  and  testing  of 
that  solution.  We  identify  a structure  for  scaffolding  case-based  learning  for  problem 
solving.  Findings  inform  design  and  instructional  practice  as  well  as  support  grounded 
theory  for  design  and  development  of  case-based  learning  environments. 


Overview 

The  research  we  report  on  is  of  design,  development,  implementation,  and  evaluation  of  an  instructional 
environment  for  teaching  and  learning  network  engineering  for  cybersecurity.  Computer  Science  programs  need  to 
prepare  the  cybersecurity  workforce  by  engaging  students  in  interactively  solving  intellectually  challenging 
cybersecurity  problems.  The  Virtual  Network  Engineering  Laboratory  (VNEL)  provides  students  with  opportunities 
for  hands-on  network  engineering  experience  by  using  a virtual  laboratory.  We  have  developed  and  disseminated  the 
online  VNEL  to  community  colleges  for  teaching  students  how  to  solve  cybersecurity  problems.  Through  the 
VNEL,  we  currently  engage  network  engineering  students  in  fifteen  underserved  community  colleges  around  the 
country  in  case-based  cybersecurity  problem-solving.  Students  focus  on  technical  solutions  in  network  management. 
The  VNEL  is  instructionally  innovative  and  differs  from  conventional  teaching  laboratories  in  that  it  does  not 
require  students  to  be  physically  located  with  the  network  equipment  they  are  learning  to  manipulate.  Instead,  the 
VNEL  offers  a technologically  innovative  environment  for  computer  engineering  instruction  in  which  students 
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remotely  manipulate  equipment  in  a real  network  and  conduct  cased-based  problem-solving  exercises  in  a 
controlled,  high-fidelity  environment  via  the  internet  using  their  web  browsers.  The  VNEL  enables  instruction  to  be 
efficiently  and  effectively  distributed  across  geographic  regions,  thereby  reaching  greater  numbers  of  students  than 
would  be  possible  through  traditional  face-to-face  or  on-site  laboratory  instruction. 

Case-based  learning  was  chosen  as  the  instructional  model  because  novices  needed  to  learn  how  to  gain  and  refine 
their  expertise  at  solving  cybersecurity  problems.  Constructivist  learning  theory  suggests  that  such  problem-solving 
is  best  learned  in  the  context  of  problem,  project,  or  case-based  learning  environments  which  provide  experiences 
that  facilitate  knowledge  construction  (Jonassen,  1999).  When  network  engineering  students  build  experience 
through  authentic  cases  they  learn  to  manage  complexity  and  think  like  network  engineers.  In  effective  learning 
environments,  experience  is  scaffolded  with  expert  case  knowledge. 

Approach:  Design  and  Development  Research 

A growing  body  of  educational  researchers  argues  that  design  and  development  research,  also  called  design-based 
research,  be  conducted  to  bridge  theoretical  research  and  educational  practice  (Collins,  A.,  Joseph,  D.,  & Bielaczyc, 
K.  2004;  The  Design-based  Research  Collective  2003).  Richey  and  Klein  (2007)  define  design  and  development 
research  as  “the  study  of  design  and  development  processes  as  opposed  to  performing  [those  processes],”  (p.  xvi). 
Such  research  generates  usable  knowledge  about  design  and  development  of  innovative  learning  environments  and 
contextualized  applications  of  theories  of  learning  and  teaching.  Explications  of  design  and  development 
experiences  in  the  real-world  provide  lessons-learned  regarding  instructional  design,  development,  and 
implementation  of  learning  environments  and  interventions  in  education. 

Design  and  development  research  follows  these  four  steps:  (1)  analysis  of  a practical  problem  by  researchers  and 
practitioners;  (2)  development  of  solutions  with  a theoretical  framework;  (3)  evaluation  and  testing  of  solutions  in 
practice;  and  (4)  documentation  and  reflection  to  inform  design  and  development  of  instruction  in  other  contexts 
(Reeves,  2000,  p.  25).  Goals  of  design  and  development  research  are  to  generate  (a)  guidance  for  both  designers  and 
practitioners,  and  (b)  theory  that  is  grounded  in  practice.  Guidance  and  theory  generated  from  design  and 
development  research  emerges  from  long-term  collaboration  among  researchers  and  practitioners. 

The  Practical  Problem 

Harnessing  the  potential  of  the  Internet  requires  that  current  and  future  generations  of  scientists,  engineers,  and 
educators  be  trained  to  design,  develop,  deploy,  and  operate  medium-  to-  large-scale  networked  cyberinfrastructure. 
The  security  of  our  nation’s  technical  infrastructure  depends  upon  individuals  and  people  running  businesses, 
particularly  small  businesses,  coming  to  understand  the  cyber  landscape  and  how  to  protect  their  investments  and 
assets  as  managed  on  computers.  Such  audiences  are  spread  across  America  and  are  served  by  community  colleges 
that  often  cannot  afford  to  design  or  run  cyber  security  programs  of  their  own.  America’s  comprehensive  strategy 
for  cyber  security  requires  training  individuals  and  small  businesses  to  secure  their  own  parts  of  cyberspace,  as  gaps 
in  the  security  of  one  group  of  cyber  participants  can  be  a conduit  through  which  other  participants  are  attacked.  In 
light  of  this,  the  need  for  training  individuals  who  manage  small  enterprises  is  particularly  acute,  as  they  frequently 
have  the  least  knowledge  of  cybersecurity,  and  thus  are  among  the  most  vulnerable. 

Unfortunately,  there  is  a dearth  of  instructors  with  expertise  in  cybersecurity,  particularly  in  rural  parts  of  our  nation. 
In  addition,  traditional  hands-on  laboratories  on  network  equipment  are  inherently  expensive,  both  in  terms  of 
capital  outlay  for  equipment  and  in  terms  of  labor  cost  for  operations  and  management.  Without  expert  instructors  or 
laboratory  equipment  for  hands-on  learning,  students,  and  subsequently  the  workforce,  are  ill  prepared  to  address 
cybersecurity  problems. 

The  Solution  Framed  in  Distributed  Cognitions  and  Cognitive  Flexibilty  Theories: 

Design  of  the  case-based  VNEL  is  framed  by  two  synergistic  learning  theories,  distributed  cognitions  theory 
(Salomon,  1997).  and  cognitive  flexibility  theory  (Spiro,  Fletovich,  Jacobson,  Coulson,  1991).  According  to 
distributed  cognitions  theory,  learning  is  driven  by  the  wish  to  solve  problems  perceived  by  the  learner,  and  ability 
to  solve  problems  is  achieved  through  real-world  experience.  Today's  instructional  designers  advocate  for  learning 
environments  where  instructors  require  students  to  engage  in  performance-based  activities  such  as  collaborative 
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problem-solving  (Jonassen,  1997).  Distributed  cognition  is  a critical  tenet  of  engaged  learning  - where  a social 
learning  community  is  energized  by  learning,  by  contributing  to  others'  learning,  and  by  personal  responsibility  for 
one’s  own  learning  and  contributions.  Engaged  learning  features  tasks  that  involve  active  cognitive  processes  such  as 
creating,  problem  solving,  reasoning,  decision-making,  and  evaluation.  Distributed  cognitions  theory  proposes  that 
“knowledge  is  commonly  socially  constructed,  through  collaborative  efforts  toward  shared  objectives...”  (Pea, 

1997).  According  to  the  theory,  intelligence  is  not  solely  an  attribute  within  individuals,  but  is  generated 
collaboratively  in  community.  Bringing  multiple  experts  into  a learning  environment  distributes  expertise  among 
learners.  Students  seek  different  forms  of  support  from  instructors,  their  peers,  colleagues,  and  other  community 
members  (mentoring,  coaching,  modeling,  mediation)  (Fosnot,  1996).  Tools  in  the  physical  environment  serve  as 
mediating  structures  that  shape  and  direct  human  activity;  and  human  activity  emerges  from  human  need.  Resources 
that  shape  and  enable  activity  are  distributed  in  configuration  across  people,  environments,  and  situations  so  that 
“intelligence  is  accomplished  rather  than  possessed”  (Pea,  1997). 

The  fundamental  premise  in  Spiro's  cognitive  flexibility  theory  (Spiro,  Feltovich,  & Coulson,  2004).  is  that  multiple 
perspectives  must  be  available  to  learners  in  a knowledge  domain  so  that  learners  can  flexibly  build  understanding. 
Well  designed  instruction  provides  conditions  that  facilitate  cognition  that  is  flexible  enough  to  transfer  to 
circumstances  other  than  those  the  student  encounters  during  instruction.  Spiro  has  shown  that  a well-designed 
hypertext  environment  can  facilitate  such  cognitive  flexibility.  First,  the  theory  emphasizes  the  importance  of 
embedding  multimodal  representations  of  to-be-learned  content  in  complex,  case-based  scenarios.  Second,  it 
recommends  that  learning  activities  provide  multiple  representations  of  content  in  order  to  support  learners  as  they 
build  mental  models.  Third,  the  theory  describes  ways  that  hypertext  instructional  environments  avoid 
oversimplification  of  content  and  support  context-dependent  knowledge.  Fourth,  cognitive  flexibility  theory 
suggests  that  knowledge  sources  be  highly  interconnected  rather  than  compartmentalized. 

Cybersecurity  content  is  presented  in  the  VNEF  in  the  format  of  case-based  learning  as  suggested  by  cognitive 
flexibility  theory  (Spiro,  Fletovich,  Jacobson,  Coulson,  1991;  Jonassen,  1997).  Case-based  learning  is  widely  used  in 
professional  education  to  engage  learners  with  real  world  situations  in  order  to  give  to  them  the  expertise  required  to 
be  successful  in  the  workforce.  We  chose  case-based  learning  as  the  instructional  approach  because  novices  need  to 
learn  how  to  gain  and  refine  their  expertise  at  solving  real-world  cybersecurity  problems.  When  cybersecurity 
students  build  experience  through  authentic  cases  they  learn  to  think  like  network  engineers.  In  effective  learning 
environments,  experience  is  scaffolded  with  expert  case  knowledge.  A case  serves  as  a representation  of  real-world 
phenomena  and  is  a safe,  yet  meaningful  context  in  which  students  can  develop  understanding  of  the  complexity  of 
network  engineering.  Jonassen  and  Hernandez-Serrano  (2002)  describe  the  case-based  reasoning  cycle  as  the 
presentation  of  a new  problem-case  to  solve.  Given  a case,  learners  apply  previous  experiences  and  general 
knowledge  to  solve  the  case,  suggest  solutions,  test  the  solutions,  revise  their  suggestions,  and  confirm  solutions. 
During  the  cycle,  learners  retrieve,  reuse,  revise,  and  retain  understanding  until  expertise  is  gained  from  the 
problem-case.  Expertise  is  derived  from  expert  scaffolding  and  experience  in  the  case's  content  domain;  in 
cybersecurity,  such  scaffolding  and  experience  are  both  hard  to  come  by  and  are  impractical  for  novices. 

Based  upon  the  two  theories  described  above,  the  VNEF  system  was  designed  to  provide  novices  with  opportunities 
to  gain  expertise  and  cognitive  flexibility  in  cybersecurity  from  multiple  experts  through  multimodal  experience  in 
the  context  of  real-world  problems.  Such  real-world  problems  are  presented  to  students  in  sequences  of  simple 
scenarios  and  increasingly  complex  cybersecurity  cases  to  be  solved.  Through  these  scenarios  and  cases  students 
build  mental  models  that  they  can  draw  upon  for  future  problem  solving.  The  VNEF  provides  for  both  distributed 
expertise  and  hands-on  experience  with  real-world  cybersecurity  problems  making  it  an  ideal  environment  for 
testing  distributed  cognitions  and  cognitive  flexibility  theories. 

Virtual  Network  Engineering  Laboratory 

The  technical  architecture  for  the  environment  is  represented  in  figure  1 . The  learner  can  experience  the 
environment  autonomously  or  with  the  help  of  an  instructor. 
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Figure  1.  The  VNEL  environment. 


We  provide  for  continuous  improvement  of  VNEL  instruction  through  implementation  cycles  of  development 
research,  evaluation,  and  revision.  Our  design  and  development  system  for  constructing  the  case-based  learning 
environment  involves  a cycle  of  rapid  prototyping  processes  that  undergo  continuous  evaluation  by  the  design  and 
development  team  and  a sample  of  users.  Priorities  for  topics  to  be  taught  in  units  of  instruction  are  set  by  the  expert 
faculty  in  community  colleges  around  the  United  States.  Cognitive  task  analyses  provide  the  foundation  for  the 
scope  and  sequence  of  each  unit  (Jonassen,  Tessmer,  & Hannum,  1999).  Based  upon  the  content  described  in  a 
given  task  analysis,  cases  are  developed  that  include  ability  to  perform  all  tasks  for  that  topic.  To  scaffold  the  ability 
to  solve  each  cybersecurity  case,  instruction  within  each  topic  is  modularized  and  scenarios  that  compose  the  tasks 
for  the  larger  case  are  developed  for  each  module.  Then  exercises  including  assessments  are  developed  for  each 
scenario. 


Figure  1.  Rapid  prototyping  process  for  the  solution. 

Ability  to  solve  cases  for  each  topic  involves  a network  technician’s  problem-solving  skill  within  that  topic.  Each 
case  presents  sufficient  detail  to  make  the  case  motivating  without  causing  students  to  focus  on  extraneous 
information.  Cases  include  a main  character,  events  and  reactions  by  the  main  character,  problems  to  solve  in  a real- 
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world  context,  a description  of  how  to  demonstrate  that  an  acceptable  resolution  to  the  problem  has  been  achieved, 
and  a call  to  solve  the  problem. 


In  the  VNEL,  students  are  initially  presented  with  a unit’s  case  and  are  told  that  upon  conclusion  of  the  unit  they 
will  have  the  necessary  skills  to  solve  it.  Each  case  presents  sufficient  detail  to  motivate  students  without  causing 
them  to  focus  on  extraneous  information.  Cases  include  a main  character,  events  and  the  main  character’s  reactions 
to  them,  problems  to  solve  in  a real-world  context,  a description  of  how  to  demonstrate  that  an  acceptable  resolution 
to  the  problem  has  been  achieved,  and  a call  to  solve  the  problem.  Students  then  work  through  a carefully  sequenced 
series  of  modules.  Each  module  contains  multiple  scenarios  and  exercises.  Within  each  exercise  are  problem 
statements,  described  prerequisites,  foundational  content,  supporting  documentation  to  complete  exercises, 
activities,  and  an  assessment.  In  the  current  design,  activities  include  performance  objectives,  directions,  the 
opportunity  to  submit  work,  and  feedback.  Ability  to  apply  the  skills  taught  in  each  exercise  is  assessed.  Assessment 
consists  of  supporting  documentation,  objectives,  directions,  student  performance  in  the  form  of  completed 
templates,  questions  answered,  or  command  line  interface  logs.  Once  students  have  successfully  completed  each  of 
the  exercises  in  a unit,  they  are  able  to  apply  the  skills  learned  in  order  to  successfully  address  the  well-structured, 
complex  problem  in  the  initial  case.  Solving  the  case  is  the  terminal  activity  in  the  unit  and  currently  serves  as  the 
students'  authentic  assessment.  Such  an  assessment  is  limited  to  providing  evidence  of  students'  abilities  to  solve  a 
well-structured  problem  as  students  have  had  prior  exposure  to  the  case  and  have  been  systematically  guided  to 
solve  that  specific  problem.  However,  the  current  instruction  does  not  prepare  students  to  solve  ill-structured 
cybersecurity  problems  as  those  they  will  encounter  in  the  workforce.  By  adding  an  automated  ITS  to  the  current 
VNEL  as  illustrated  below,  we  hope  to  prepare  students  to  approach  ill-structured  problems  from  a stance  of 
cognitive  flexibility  and  test  whether  the  cyberlearning  environment  indeed  prepares  students  for  ill-structured 
problem  solving.  That  is,  we  will  ask,  do  students  learn  the  process  of  solving  ill-structured  cybersecurity  problems 
in  the  learning  environment  by  learning  how  to  solve  a series  of  similar  structured  problems? 
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Figure3.  Structure  and  scaffolding  in  the  case-based  learning  environment  with  intelligent  tutoring  system  functions 
highlighted. 
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The  unit  on  Firewalls  serves  as  an  example.  In  order  to  address  cybersecurity  issues  with  firewalls,  network 
engineers  need  to  be  able  to  align  a topology  with  network  policy  requirements,  restrict  access,  and  configure 
networks  through  logging,  encryption,  and  VPN’s.  The  full  range  of  skills  required  for  addressing  the  complex  case 
are  described  in  Figure  4.  Once  students  have  mastered  each  of  the  skills,  they  should  be  able  to  address  the  case. 


Figure  4.  Firewalls  unit  map. 

Evaluation  and  Testing 

The  researchers  use  formative  evaluation  as  the  research  method,  continuously  investigating  the  VNEL  in  pursuit  of 
establishing  design  and  development  theory  grounded  in  experience  (Orrill,  Hannifin,  and  Glazer,  2003;  Wang  & 
Hannafin,  2005).  In  design-based  research  such  as  this,  researchers  hybridize  instructional  design  methods  with 
research  methods  as  they  investigate  their  own  processes.  Each  case,  module,  scenario,  and  exercise  was  constructed 
through  a process  involving  eight  iterations  with  the  design  team  and  periodic  one-on-one  and  small  group 
evaluation  with  the  target  audience.  Field  testing  is  scheduled  for  Winter  of  2010. 

Participants 

The  team  of  developers  includes  instructional  designers,  network  engineers,  computer  scientists,  and  network 
engineering  instructors  who  meet  and  correspond  weekly  to  inform  design  decisions.  One-on-one  and  small  group 
evaluations  were  conducted  with  11  students  at  Del  Mar  Community  College  in  Corpus  Christi,  Texas.  Two  of  the 
students  were  female  and  nine  were  male.  Two  students  were  White  and  nine  were  Hispanic.  Field  evaluations  are 
scheduled  for  the  upcoming  semester. 
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Data  Collection,  Documentation,  and  Analysis 

The  researchers  collected  and  will  continue  to  collect  data  from  design  documentation,  pretest  and  posttest 
comparisons,  usability  and  attitude  surveys,  and  faculty  and  student  focus  groups.  We  ask  questions  such  as-  Is  the 
activity  authentic?  Is  the  activity  something  a network  engineer  would  really  encounter  on  the  job?  Does  the 
terminology  used  reflect  everyday  terminology  used  by  instructors  and  practitioners  alike?  Does  the  activity  reflect  a 
competency  necessary  for  understanding  and  mastering  the  content?  Does  the  module  combine  enough  or  too  much 
information  into  one  exercise?  Design  documentation  involves  continuous  records  of  tasks  to  be  done,  task  due 
dates,  responsible  party,  and  task  status.  In  addition,  the  lead  designers  keep  journals  that  include  lessons  learned 
during  the  process.  Documentation  is  constructed  weekly.  Questions  about  content,  design,  technical  quality,  and 
attitudes  toward  the  instruction  are  delivered  in  multiple  usability  surveys,  and  focus  groups.  The  hundreds  of  inputs 
from  data  continuously  inform  revision  of  the  VNEL. 

Results 

Preliminary  findings  indicate  that  the  case-based  learning  environment  provides  students  with  a meaningful  learning 
experience  that  uniquely  prepares  them  for  the  workforce.  The  majority  of  students  were  able  to  describe  the 
problem  that  was  to  be  solved  in  the  case  and  in  the  scenarios.  Ninety  percent  felt  that  the  scenario  and  case  were 
applicable  to  something  they  might  encounter  in  an  on-the-job  situation.  Users  were  able  to  pinpoint  unclear 
language  for  revision  and  where  in  the  environment  they  needed  more  instruction.  An  important  input  from  users 
was  the  need  for  clear  connections  between  the  contents  of  modules  and  the  ultimate  case.  Student's  responses  to  a 
Likert  scale  survey  instrument  indicated  that  most  agreed  that  their  experience  using  the  VNEL  system  was 
interesting,  navigation  through  the  interface  was  easy  to  understand  and  easy  to  do,  and  the  case->  module  - 
>scenario->exercise  configuration  was  easy  to  follow.  Pretests  scores  reveal  the  need  for  instruction  that  facilitates 
problem  solving  rather  than  learning  of  discrete  skills. 

Both  instructors  and  students  appreciate  the  flexibility  regarding  sequence  and  participation  provided  in  the 
environment.  Input  from  instructors  indicated  the  need  for  an  instructors’  manual.  Instructors  also  indicated  that  they 
may  need  scaffolding  beyond  what  is  currently  provided  in  the  environment  in  order  to  support  students  of  various 
abilities.  Some  students  find  too  much  guidance  in  the  environment  while  others  express  the  need  for  more. 
Interdisciplinary  team  meetings  result  in  strategic  improvement  of  the  environment  and  provide  graduate  students  in 
both  Computer  Science  and  Instructional  Design  with  rich  professional  experience. 

Students  reported  that  they  hoped  to  get  a job  in  network  engineering  in  their  futures.  Most  importantly,  the  target 
audience  felt  that  they  were  provided  with  useful  training  to  help  them  get  a job  in  network  engineering  and  that 
they  were  doing  the  real  work  of  a network  engineer  technician  while  in  the  VNEL  system. 

Our  experience  suggests  that  small  scenarios  scaffold  more  complex  cases,  especially  when  coupled  with  tutoring 
strategies  such  as  guiding  questions,  clarification,  hints,  examples/nonexamples,  and  redirection  to  support  students’ 
success  and  cognitive  flexibility. 

Educational  Implications 

The  process  used  for  developing  the  VNEL  case-based  learning  environment  illustrates  how  instructional  designers 
can  address  societal  problems  through  case-based  learning  environments.  Over  the  years  educational  research  has 
been  criticized  for  having  little  practical  relevance.  Reeves  (2000)  calls  for  socially  responsible  design  and 
development  research  that  addresses  relevant,  real-world  problems.  Our  study  provides  an  elaborate  illustration  of 
the  process  for  conducting  such  research  in  the  field  of  educational  technology  and  instructional  design.  The  product 
under  study,  the  VNEL,  stands  as  a solution  to  substantial  societal  problems  of  lack  of  hands-on  instruction  for 
training  the  network  engineering  workforce. 

The  study  examines  implementation  of  a web-based  technology  in  a real-life  context,  thereby  testing  distributed 
cognitions  learning  theory.  “Research  examining  the  concept  of  affordances  [of  technological  tools]  is  critical  if  we 
are  to  build  a science  of  distributed  intelligence  and  a more  flexible  design  orientation  to  the  practices  of  education” 
(Pea,  1997,  p.  51).  Design  of  the  VNEL  environment  was  based  on  philosophical  assumptions  of  social 
constructivism  and  on  distributed  cognitions  theory  with  the  intention  of  supporting  collaborative  knowledge 
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construction  among  members  of  a learning  community.  The  qualitative  methods  used  to  investigate  the  overall 
impact  of  the  website  will  extend  the  research  on  use  of  technologies  to  support  constructivist  learning. 

Findings  in  this  study  will  inform  design  and  support  grounded  theory  for  design  and  development  of  case-based 
learning  environments.  Design-development  guidelines  generated  will  be  evidence -based  and  generalizable  to  other 
design  efforts  because  they  will  be  based  upon  reflective  process  analysis  rather  than  the  summative  effects  of  a 
contextualized  intervention.  They  will  be  of  practical  use  for  developers  and  could  not  be  generated  through 
traditional  experimental  research  approaches. 
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A Case  for  Stealth  Educational  Games 


Yolanda  Debose  Columbus  and  Lauren  Cifuentes 
Texas  A&M  University 

Abstract 

The  purpose  of  this  paper  is  to  make  a case  for  using  stealth  educational  games  to  increase  science  self- 
efficacy.  To  make  the  case,  the  authors  describe  a theoretical  framework  to  support  two  common  held  views:  1) 
today’s  students  are  different  from  previous  generations  and  2)  educational  games  are  sometimes  neither  engaging 
nor  educational.  The  theoretical  framework  is  also  used  to  explain  why  stealth  educational  games  are  a possible 
solution.  The  framework  includes  three  theories  and  one  empirical  study:  attention  theory,  flow  theory,  self-efficacy 
theory,  and  an  implicit  learning  study. 

The  theories  highlight  the  relationship  between  personal  experience,  attention,  self-efficacy,  and  learning 
objectives.  The  theories  affirm  that  recurring  experiences  influences  the  development  of  attention  and  self-efficacy. 
The  empirical  study  suggests  that  experience  is  also  impacted  by  explicit  learning  objectives.  The  relationship 
between  the  constructs  in  the  theory  suggests  that  for  students  with  low  self-efficacy,  sharing  objectives  may 
negatively  impact  their  experience.  Using  attention  theory,  flow  theory,  self-efficacy  theory,  and  an  implicit  learning 
study,  the  researchers  argue  that  a stealth  educational  game  will  support  positive  experience  that  can  build  science 
self-efficacy. 


Although  explicit  learning  objectives  are  thought  to  be  core  elements  of  effective  instruction,  the  value  of 
explicit  learning  objectives  has  not  been  supported  by  research  (Morrison,  Ross,  & Kemp,  2004).  Thus,  implicit 
learning  objectives  may  also  be  a core  element  of  effective  instruction.  For  over  3000  years  cultures  in  the  east  have 
used  implicit  learning  objectives  to  teach  history  and  values  while  for  the  past  30  years  instructional  design  practices 
in  the  United  States  have  focused  on  making  learning  objectives  explicit  (Marzano,  1998).  One  of  the  oldest  and 
most  widely  read  books  that  utilizes  implicit  learning  objectives,  for  example,  is  the  Bible. 

We  advocate  the  use  of  implicit  learning  objectives  rather  than  explicit  learning  objectives  in  science 
educational  games  to  improve  science  self-efficacy.  Traditionally,  educational  games  use  explicit  learning  objectives 
and  focus  on  educating  the  player.  In  contrast,  stealth  educational  games  have  implicit  learning  objectives  and  focus 
on  entertaining  the  player. 

Stealth  educational  games  are  defined  based  on  four  attributes: 

1 . Primarily  focusing  on  entertaining  the  player, 

2.  Not  informing  players  of  learning  objectives, 

3.  Hiding  tasks  that  support  the  learning  objectives  in  game  play,  and 

4.  Requiring  players  to  apply  rules  in  complex  multivariate  systems. 

The  first  attribute  contributes  to  engaging  today’s  adolescents.  The  second  and  third  attribute  address  engagement 
for  unmotivated  or  struggling  students  by  hiding  the  educational  aspects  of  the  game.  The  fourth  and  last  attribute 
four  adheres  to  one  of  the  recommendations  of  the  National  Assessment  of  Educational  Progress  (NAEP).  NAEP 
urges  that  science  curriculum  teach  students  about  complex  multivariate  systems: 

“Students  should  be  able  to  identify  and  define  the  system  boundaries  [i.e.  know 
rules],  identify  the  components  [i.e.  multiple  variables],  and  their  interrelationships 
[e.g.  complex  relationships],  and  note  the  inputs  and  outputs  of  the  systems  [i.e.  know 
and  apply  rules]  (O'  Sullivan  & Weiss,  1998,  p 309)” 

In  today’s  era  of  accountability  and  standard-based  assessment,  explicit  learning  objectives  have  become 
commonplace  in  science  curriculums  (Scruggs  & Mastropieri,  2007).  Unfortunately,  this  approach  has  not  had  the 
anticipated  impact  (Lee  & Wong,  2004).  In  an  effort  to  improve  the  impact  of  explicit  learning  objectives,  some 
educators  began  incorporating  explicit  learning  objectives  in  games,  including  educational  science  games. 
Regrettably,  educational  games  could  not  and  did  not  tackle  the  problem.  The  reason  for  the  mediocre  impact  has 
been  explained  by  two  common  arguments: 


84 


1.  “Today’s  students  think  and  process  information  fundamentally  differently  from  their  predecessors 
(Prensky,  2001,  pi).” 

2.  All  too  often,  educational  games  are  neither  entertaining  nor  educational  (Papert,  1998). 

Since  these  two  arguments  have  been  firmly  acknowledged  by  numerous  experts  in  the  field  (e.g.  Chen,  2007;  Gee, 
2003;  Prensky,  2001,  Van  Eck,  2006,  Tapscott,  1998),  we  focus  on  synthesizing  three  theories  and  one  empirical 
study. 

We  support  these  two  arguments  using  the  selective  attention  construct  from  attention  theory,  flow  theory, 
self-efficacy  theory,  and  an  implicit  learning  study  conducted  by  Berry  and  Broadbent  (1998)  (see  Figure  1). 
Selective  attention  construct  describes  the  relationship  between  repeated  practice  and  focused  attention.  Focused 
attention  along  with  flow  (e.g.  engagement)  and  learning  objectives  influences  personal  experience.  Personal 
experience,  in  turn,  directly  impacts  the  development  of  self-efficacy,  including  science  self-efficacy.  Results  from 
the  implicit  learning  study  suggest  that  implicit  learning  objectives,  rather  than  explicit  learning  objectives,  in 
complex  systems  help  performance. 


Implicit  learning  (Berry  & Broadbent,  1998) ^ 

Flow  theory  (Chen,  2004)  . _ . 

Self-efficacy  theory  (Bandura,  1986)  ► 

Selective  Attention  Theory  (Plude,  Enns,  & Brodbeur,  1994)  ^ 

Figure  1.  Theoretical  Framework 


Argument  1 

The  expectations  and  skills  of  this  generation  are  very  different  from  the  baby  boomers  (Prensky,  2001; 
Tapscott,  1998).  During  their  formative  years,  digital  and  interactive  technology  surrounded  today’s  adolescents. 
According  to  A.  C.  Nielsen  ratings  in  2006,  the  average  child  watched  1,680  minutes  of  television  a week. 
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“The  typical  8-18  year  old  lives  in  a home  with  3.6  CD  or  tape  players,  3.5  TVs, 

3.3  radios,  2.9  VCRs/DVDs  players,  2.1  video  game  consoles,  and  1.5 
computers  according  to  a 2005  study  by  the  Kaiser  Family  Foundation.  .The 
Kaiser  study  found  that  64%  of  young  people  ages  8-18  have  downloaded  music 
from  the  internet;  66%  use  instant  messaging;  39%  have  a cell  phone;  32%  have 
created  a personal  website  or  web  page;  18%  have  an  MP3  player;  and  13% 
have  a handheld  device  that  connects  to  the  Internet  (Macarthur  Foundation, 

2005,  p??).” 

Adolescents’  digital  technology  experiences  include  a significant  amount  of  practice  playing  video  games. 
In  2005  “about  83  percent  of  young  people  between  the  ages  of  8 and  18  play[ed]  video  games  regularly” 

(Macarthur  Foundation,  2005,  p.  9).  In  2008,  ninety-seven  percent  of  adolescents  between  the  ages  of  12-17,  self- 
reported  that  they  played  computer,  web,  portable,  or  console  games  regularly  (Lenhart,  et.  al,  2008).  It  is  safe  to  say 
that  video  games  are  a popular  past  time  of  adolescents. 

Adolescents  have  repeated  practice  playing  video  games.  According  to  attention  theory,  repeated  practice 
impacts  the  development  of  selective  attention  (Plude,  Enns,  & Brodbeur,  1994).  Selective  attention  is  the  process  of 
focusing  attention  on  relevant  information  and  disregarding  irrelevant  information  (Plude,  Enns,  & Brodeur,  1994). 
Thus,  repeated  practice  in  video  games  impacts  process  adolescents  use  to  select  relevant  information. 

Selective  attention  makes  use  of  characteristics  internal  and  external  to  the  person.  Internal  characteristics 
include  items  such  as  priorities,  values,  and  goals.  External  characteristics  include  items  such  as  color  and  sound. 
When  playing  computer  games  repeatedly,  a person's  priorities,  values,  and  goals  (e.g.  internal  characteristics) 
become  associated  with  the  sights,  color,  and  sound  (e.g.  external  characteristics)  in  computer  games.  For  example, 
a person  who  performs  well  in  a computer  game  may  associate  confidence,  excitement,  and  competence  (e.g. 
internal  characteristics)  with  vivid  color  and  deafening  sound  (e.g.  external  characteristics).  As  a result,  “today’s 
students  think  and  process  information  fundamentally  differently  from  their  predecessors  (Prensky,  2001,  pi).” 

Argument  2 

All  too  often,  educational  games  are  neither  engaging  nor  educational  (Papert,  1998).  An  educational  game 
typically  does  not  facilitate  flow.  Flow  describes  the  situational  attributes  that  lead  to 

• Feelings  of  immersion, 

• A merging  of  awareness  to  current  actions, 

• A high  degree  of  concentration, 

• Self-confidence,  and 

• Feelings  of  control  (Eccles  &Wigfield,  2002). 

These  five  situational  attributes  influence  a player’s  degree  of  engagement  during  an  activity,  such  as  playing 
computer  games.  Entertainment  computer  games  are  purposely  designed  to  facilitate  flow.  Since  adolescents  spend 
a considerably  amount  of  time  playing  entertainment  video  games,  they  expect  similar  flow  experiences  when 
playing  educational  video  games  (Prensky,  2005). 

Adolescents  flow  expectations  are  impacted  by  their  selective  attention.  As  mentioned  previously,  selective 
attention  refers  to  the  process  of  selecting  relevant  information  and  disregarding  irrelevant  information,  which 
develops  as  a result  of  repeated  experiences  (Plude,  Enns,  & Brodbeur,  1994).  Through  repeated  practice  in 
entertainment  games  adolescents  are  programmed  to  automatically  focus  their  attention  on  game  elements  that  lead 
to  flow  (e.g.  a feeling  of  immersion).  Because  the  purpose  of  educational  games  is  instruction  and  not  facilitating 
flow,  flow  is  often  a secondary  goal  or  a by-product.  Educational  games  that  superficially  mimic  the  environment  of 
entertainment  games  may  not  facilitate  the  flow  experience  adolescents  are  expecting.  Not  meeting  these 
expectations  leads  to  educational  games  that  are  not  engaging  to  adolescents. 

Educational  games  have  not  had  the  anticipated  impact  on  learning  (Van  Eck,  2006).  Educational  games 
are  designed  by  educators  or  instructional  designers.  Unfortunately,  these  professionals  are  often  not  familiar  with 
the  gaming  environment;  thus,  they  do  not  know  how  to  best  use  the  game  elements  (e.g.  storyline,  character 
development)  to  support  learning  (Papert,  1998;  Van  Eck,  2006).  This  limited  experience  is  confounded  when  trying 
to  create  educational  games  to  teach  science.  Science  involves  applying  rules  using  multiple  variables  that  are 
interrelated  (O’  Sullivan  & Weiss,  1998).  Performance  in  these  complex  multivariate  systems  may  be  impeded  by 
explicit  learning  objectives,  typically  found  in  educational  games  (Berry  & Broadbent,  1988). 

For  learners  who  struggle  in  science,  explicit  learning  objectives  in  science  may  hinder  performance.  Berry 
and  Broadbent  ( 1988)  report  that  explicit  instructions  hindered  performance  in  a complex  multivariate  system.  In 
their  study,  the  complex  multivariate  system  was  defined  by  the  relationship  between  the  computer  response  and 
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participants;  the  computer  response  depended  on  participant’s  input  in  preceding  trials.  In  this  complex  system  of 
interrelated  variables,  participants  performed  better  when  the  instructions  and  learning  objectives  were  implicit 
rather  than  explicit.  One  explanation  for  their  results  can  be  attributed  to  the  selective  attention  construct  and  self- 
efficacy  theory. 

Recall  that  selective  attention  becomes  automatic  through  repeated  practice  (Plude,  Enns,  & Brodeur, 

1994).  During  recurring  negative  experiences  in  science  class,  students  may  learn  to  selectively  attend  to  goals, 
objects,  or  items  not  related  to  science.  For  example,  if  a student  feels  foolish  during  science  class,  he  or  she  may 
focus  on  entertaining  the  class  instead  of  the  learning  objectives.  Through  repeated  practice  the  struggling  student 
may  learn  to  avoid  science.  Anything  related  to  science  may  automatically  cause  a similar  response.  This  lack  of 
focus  will  impede  performance. 

Similarly  to  selective  attention  theory,  recurring  experience  also  impacts  the  development  of  self-efficacy. 
Self-efficacy  is  an  individual’s  belief  in  their  ability  to  accomplish  a goal  or  task  in  a specific  domain  or  context 
(e.g.  science  self-efficacy;  Bandura,  1986).  Since  personal  experience  is  a dominant  source  of  self-efficacy, 
recurring  negative  personal  experience  can  create  low  self-efficacy  (Bandura,  1986).  For  example,  students  who 
struggle  in  science  class  may  develop  low  science  self-efficacy. 

There  is  a circular  relationship  between  personal  experience,  self-efficacy,  and  effort  ( Bandura,  1986; 
Pajares,  1996).  Personal  experience  directly  influences  self-efficacy.  In  turn,  self-efficacy  is  directly  related  to  effort. 
Fow  science  self-efficacy  may  create  little  effort  in  science  activities.  The  circle  is  completed  by  the  relationship 
between  effort  and  personal  experience;  effort  has  a direct  impact  on  personal  experience.  So,  little  effort  may  create 
a negative  personal  experience.  For  struggling  or  unmotivated  learners,  this  circular  relationship  between  personal 
experience,  self-efficacy  and  effort  may  reinforce  low  self-efficacy.  This  leads  one  to  question  how  then,  can  low 
self-efficacy  be  improved  in  struggling  or  unmotivated  learners.  The  answer  lies  in  the  definition  of  self-efficacy. 

According  to  the  definition,  self-efficacy  is  domain  and  context  specific  (Bandura,  1986).  Domain  and 
context  are  characterized  by  attributes  of  the  environment  and  the  activity.  If  the  attributes  of  the  environment  or 
activity  triggers  low  self-efficacy,  the  process  of  reinforcing  low  self-efficacy  may  began.  One  way  to  not  trigger 
low  self-efficacy  is  to  put  the  person  in  an  environment  (e.g.  video  games)  that  triggers  high  self-efficacy.  Since  the 
goal  is  to  improve  low  self-efficacy,  the  environment  must  also  contain  attributes  that  are  related  to  the  domain  or 
context  in  which  the  person  has  low  self-efficacy.  In  essence,  the  true  purpose  of  the  activity,  for  example, 
improving  low  science  self-efficacy,  is  hidden  from  the  person  by  the  attributes  of  the  environment.  Stealth 
educational  games  fulfill  this  criterion  because,  similarly  to  entertainment  games,  their  primary  focus  is  entertaining 
the  player  and  they  also  include  implicit  learning  objectives. 

Opponents’  arguments 

Opponents  may  base  their  argument  on  instructional  design  principles  (Smith  & Ragan,  2005)  or  the 
domain  and  context  specificity  of  self-efficacy.  Instructional  design  principles  propose  that  making  the  learning 
objectives  explicit  is  beneficial  because  explicit  learning  objectives  prepare  the  learners  to  leant.  Since  self-efficacy 
is  domain  and  context  specificity,  opponents  may  argue  that  building  self-efficacy  in  a game  will  not  transfer  to 
activities  outside  the  game. 

Instructional  design  principles  recommend  that  after  gaining  the  learners  attention,  instruction  should  share 
the  learning  objectives  (Gagne,  1977;  Smith  & Ragan,  2005).  The  argument  follow  that  by  sharing  the  learning 
objectives,  the  learner  will  recall  related  prior  knowledge.  Thus,  knowing  the  goal  will  help  structure  their  learning 
experience  and  knowledge  (Morrison,  Ross,  Kemp,  2004).  The  benefits  of  explicit  learning  objectives  are  only 
available  to  motivated  or  prepared  learners.  As  noted  by  the  Reber  (1993),  explicit  instructions  or  sharing  the 
learning  objectives  could  have  a positive  but  also  could  have  a negative  impact  on  learners.  Fearners  who  are  not 
interested  or  who  struggle  with  the  content  may  choose  to  avoid  the  learning  activities  or  give  less  effort. 

As  for  the  dispute  based  on  self-efficacy  theory,  most  of  the  self-efficacy  research  focuses  on  one 
dimension  of  self-efficacy  - strength  (Cervone,  2000)  but  Bandura  (1977)  describes  three  dimensions  of  self- 
efficacy  - magnitude,  strength,  and  generality.  Magnitude  describes  the  difficulty  level  of  the  task.  Strength  reflects 
the  certainty  of  the  belief.  Generality  refers  to  the  transferability  to  other  context  and  tasks.  Generalizations  in  self- 
efficacy  depend  on  self-knowledge  that  comes  to  mind  in  various  situations  and  settings  (Cervone,  2000). 

Through  repeated  practice  and  practice  variability  self-efficacy  is  generalized  (Holladay  & Quinones,  2003; 
Pugh  & Bergin,  2006).  In  addition,  self-efficacy  is  increased  most  significantly  by  personal  experience  (Bandura, 
1986).  By  placing  the  personal  experience  in  a stealth  educational  game,  learners  are  more  likely  to  play  the  game 
repeatedly.  By  definition  stealth  educational  games  include  complex  systems  with  interrelationships  between 
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multiple  variables,  thus,  repeatedly  playing  a stealth  educational  game  provides  practice  variability.  Through 
repeated  practice  and  practice  variability  self-efficacy  developed  in  the  game  should  generalize. 


Discussions  and  Implications 

The  two  arguments  presented  in  this  paper  are  primarily  learner-centered.  A significant  portion  of  today’s 
adolescents  developed  cognitively  in  a digital  environment.  Thus,  the  way  today’s  adolescents’  process  information 
as  well  as  what  motivates  and  engages  them  is  different  from  previous  generations.  As  a result  of  these 
characteristics,  the  instructional  approaches  typically  used  may  be  contributing  to  science  achievement  gap  among 
various  ethnicities  and  socio-economic  levels. 

For  unmotivated  and/or  struggling  students  repeated  negative  experiences  in  the  classroom  may  reinforce 
low  academic  self-efficacy.  Self-efficacy  is  important  because  it  affects  goals,  interests,  and  choices,  which  in  turn 
impacts  career  paths  and  opportunities.  Without  an  intervention  inside  and  outside  of  school  these  students  will 
have  a difficult  time  changing  their  path. 

Using  a stealth  educational  game  to  impact  academic  self-efficacy  gives  us  new  avenues  for  helping 
unmotivated  and/or  struggling  students.  Since  stealth  educational  games  include  a focus  on  entertainment,  they  may 
appeal  to  students  outside  of  school.  Thus,  these  games  are  not  just  limited  to  the  classrooms;  these  games  may  be 
sold  in  stores,  integrated  into  youth  development  programs,  or  made  available  on  the  Internet.  They  could  become  a 
part  of  adolescents'  daily  entertainment  package.  Thus,  academic  self-efficacy  may  be  increased  in  a myriad  of 
context. 


Conclusions 

Instructional  design  principles  suggest  that  well-designed  instruction  shares  learning  objectives  with  the 
learners  (Smith  & Ragan,  2005).  Counter  to  instructional  design  principles,  implicit  learning  research  suggests  that 
explicit  learning  objectives  can  also  negatively  impact  performance.  Based  on  the  arguments  in  this  paper,  the 
appropriateness  of  sharing  learning  objectives  may  not  be  clear-cut.  Rather,  the  appropriateness  may  depend  on 
learner  and  context  analysis. 
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Abstract 

This  study  examined  the  extent  to  which  the  modality  principle  of  multimedia  learning  extends  to  learning  from  a 
static  diagram  and  associated  hypertext.  Participants  were  randomly  assigned  to  2 modality  conditions  (visual  text  vs. 
spoken  text)  and  2 information  cueing  conditions  (cueing  vs.  no  cueing).  The  results  showed  a main  effect  for 
modality,  revealing  a reverse  modality  effect.  Participants  studying  visual  text  performed  better  than  those  studying 
spoken  text  on  three  separate  dependant  measures.  These  results  provide  insight  into  the  boundaries  that  should  be 
applied  to  the  modality  principle. 


Theoretical  Framework  and  Hypothesis 

The  modality  principle  in  multimedia  learning  has  received  considerable  research  support  over  the  last  few  decades 
(Ginns,  2005;  Mayer,  2001;  Sweller,  van  Merrienboer,  & Paas,  1998).  This  principle  states  that  learning  from  words 
and  pictures  is  improved  when  written  or  on-screen  text  is  replaced  with  spoken  text.  A theoretical  rationale  for  this 
principle  is  provided  by  Baddeley’s  (1992)  model  of  working  memory.  According  to  Baddeley,  working  memory 
contains  two  sub-systems,  one  for  processing  visual  information  and  another  for  processing  acoustic  information. 
Presenting  textual  information  visually  (as  on-screen  text)  is  purported  to  overload  the  visual  subsystem  during 
multimedia  learning  due  to  the  need  to  process  both  pictorial  (e.g.,  animation)  and  textual  information  within  the 
same  memory  subsystem.  However,  according  to  the  modality  principle,  this  unimodal  presentation  format  can  be 
improved  by  employing  a bimodal  format,  wherein  textual  information  is  presented  auditorily  and  pictorial 
information  is  presented  visually.  This  presentation  format  is  purported  to  reduce  cognitive  load  by  using  the  total 
capacity  of  working  memory  more  efficiently  (Tabbers,  Martens,  & van  Merrienboer,  2004).  The  modality  principle 
has  been  linked  to  reduced  mental  effort  and  study  time  during  instruction  and  to  improved  performance  on 
retention,  transfer,  and  matching  tests  (Tabbers  et  al.,  2004). 

Notwithstanding  the  research  support  for  the  modality  principle,  recent  research  has  shown  that  the  principle  may 
not  apply  to  all  multimedia  learning  situations.  For  example,  Tabbers  et  al.  (2004)  found  that  a visual  presentation  of 
text  was  superior  to  a spoken  presentation  when  learners  were  given  control  over  the  pacing  of  the  instruction.  In 
another  study,  Wouters,  Paas,  & van  Merrienboer  (2009)  found  that  when  learners  were  prompted  to  study  visual 
text  they  performed  better  on  a transfer  test  than  when  they  received  no  prompts,  suggesting  that  learner  inattention 
to  visual  text  may  be  an  important  factor  in  prior  studies  of  the  modality  principle.  There  is  also  evidence  that  visual 
text  may  be  superior  to  spoken  text  if  the  subject  matter  pertains  to  learning  spatial  relations  (Penny,  1989). 

In  the  current  study  we  examined  the  generalizability  of  the  modality  principle  to  an  instructional  situation  that  prior 
research  suggests  may  be  conducive  to  a reverse  modality  effect.  That  is,  where  on-screen  text  is  superior  to  spoken 
text.  To  create  this  situation  the  following  elements  were  incorporated  into  the  treatment  materials:  1)  learner  control 
of  instructional  content,  2)  a learning  task  with  a significant  spatial  component,  and  3)  information  cueing  to  draw 
attention  to  specific  features  within  the  learning  materials.  This  is  the  first  time  these  three  elements  have  been 
explored  together  in  the  context  of  studying  the  modality  principle.  Our  hypothesis  was  that  learners  studying  on- 
screen text  would  outperform  those  studying  spoken  text  due  to  the  presence  of  learning  conditions  favorable  to  a 
reverse  modality  effect  in  multimedia  learning. 
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Method 


This  study  investigated  the  effects  of  modality  and  information  cueing  on  learning  from  a computer-based  diagram 
and  related  text.  Modality  (visual  text  vs.  spoken  text)  and  information  cueing  (cueing  vs.  no  cueing),  see  Figure  1, 
were  examined  between  subjects  to  form  a 2 X 2 factorial  design.  One  hundred  and  fifty-two  students  from  a large 
southwestern  university  were  randomly  assigned  to  one  of  the  four  experimental  conditions.  The  dependant 
measures  consisted  of  a matching  test,  a comprehension  test,  and  a diagram  reconstruction  test.  All  dependant 
measures  were  delivered  via  computer.  Students  received  credit  toward  their  final  course  grade  for  participation. 
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Figure  1.  Four  different  instructional  units  in  terms  of  modality  and  information  cueing 

The  matching  test  consisted  of  a listing  of  the  12  places  of  articulation  (e.g.  Dental)  and  a corresponding  listing  of 
the  sounds  made  at  each  place  (e.g.,  “th”  as  in  thunder ),  plus  three  distracters.  The  participants  were  required  to 
match  each  place  with  its  corresponding  sound.  The  comprehension  test  was  a 12-item  instrument  designed  to 
measure  student  comprehension  of  the  computer  program.  Finally,  the  reconstruction  test  presented  each  participant 
with  a blank  outline  of  the  Places  of  Articulation  diagram  and  a listing  of  the  12  places  of  articulation  next  to  the 
diagram.  Each  participant  was  required  to  drag  the  name  of  each  place  to  its  correct  location  on  the  diagram. 

The  experimental  materials  consisted  of  a computer-presented  diagram  depicting  12  places  of  articulation  in  human 
speech  with  a corresponding  324  word  expository  text.  The  text  contained  12  sentences  with  each  sentence 
hyperlinked  to  a specific  place  of  articulation  on  the  diagram.  The  participants  accessed  the  text  by  selecting  a place 
of  articulation  on  the  diagram  with  a computer  mouse.  In  the  visual  text  condition  each  sentence  was  presented  as 
on-screen  text;  in  the  spoken-text  condition  the  text  was  presented  as  narration  (i.e.,  spoken  text).  In  the  cueing 
condition  an  animated  pointer  directed  the  participants’  attention  to  a selected  place  of  articulation;  In  the  no-cueing 
condition  there  was  no  pointer.  The  participants  had  complete  control  over  the  pace  and  sequence  with  which  they 
selected  the  hyperlinked  text. 
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Students  participated  in  the  experiment  in  a computer  lab  with  15  to  18  participants  in  each  experimental  session. 
They  were  instructed  to  study  the  diagram  and  the  related  text  for  10  minutes.  After  10  minutes  of  study  and  a one 
minute  rest  period,  the  participants  were  given  the  matching,  comprehension,  and  reconstruction  measures. 

Results  and  Conclusions 

ANOVA  revealed  main  effects  on  the  modality  variable  for  matching  F(  1,151)  = 6.10,  p < .05,  comprehension 
F(  1,151)  = 6.63,  p < .05,  and  reconstruction  test  performance  F(  1,151)  = 7.30,  p < .01.  The  means  for  visual  text 
were  higher  than  the  means  for  spoken  text  on  all  three  dependant  measures.  No  main  effects  were  observed  on  the 
cueing  variable,  and  there  was  no  modality  by  cueing  interaction  effect.  In  summary,  we  observed  a reverse 
modality  effect  for  all  dependant  measures,  as  hypothesized. 

The  results  of  this  study  support  the  hypothesis  that  modality  effects  are  not  likely  to  occur  with  diagrams  and  text 
when  learners  have  control  over  the  pacing  and  sequence  of  their  study  and  spatial  learning  is  an  important 
instructional  outcome.  In  fact,  we  observed  a reverse  modality  effect  favoring  on-screen  text  over  spoken  text. 

Since  cueing  was  not  a significant  factor  in  this  study  we  cannot  attribute  the  reverse  modality  effect  to  increased 
attention  to  the  written  text  ( Wouters  et  al.,  2009).  Rather,  the  best  explanation  appears  to  be  that  user  control  and 
spatial  learning  are  conditions  favorable  to  learning  from  on-screen  text.  These  results  will  guide  practitioners 
during  the  design  and  development  of  multimedia  materials  and  to  theorists  as  they  seek  to  clarify  the  boundaries  of 
the  modality  effect  in  multimedia  learning. 
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The  popularity  of  Online  Social-Networking  Websites  (OSNW)  such  as  MySpace.com  and  Facebook.com 
has  grown  considerably  throughout  recent  years  (Gosling,  Gaddis  & Vazire,  2007).  The  use  of  Internet-based 
medians  such  as  blogs,  personal  websites,  and  virtual  communities  by  college  students  have  continued  to  increase  in 
recent  years  and  have  impacted  how  these  students  communicate  with  each  other  in  both  online  and  offline  contexts 
(Pempek,  Yermolayeva  and  Calvert,  2009).  Recent  high  school  graduates  spent  a substantial  time  of  their 
development  in  the  digital  age  and  have  been  labeled  “digital  natives”  because  of  the  ease  in  which  they  operate  in 
these  virtual  worlds  and  formats  (Prensky,  2001). 

Sites  such  as  Facebook  and  MySpace  are  accessible  to  most  internet  users  and  anyone  with  an  email 
address.  Users  can  provide  information  through  creating  a ‘profile’  with  the  intent  of  communication  with  others, 
meeting  new  friends  and  connecting  with  old  friends.  In  addition  to  connecting  to  friends,  dating  use,  career 
searches,  feedback  and  blogging,  users  post  self-descriptive  information. 

Facebook  is  merely  four  years  old,  but  already  boasts  of  “More  than  68  million  active  users”  and  “An 
average  of  250,000  new  registrations  per  day  since  Ian.  2007”,  {Facebook. com  website,  2008).  We  chose  Facebook 
due  to  its  organizational  structure.  Website  users  designate  themselves  with  a university  affiliation  along  with  other 
personal  information  such  as  gender,  musical  tastes,  courses  taken,  education  background,  etc.  Facebook  users  can 
also  post  picture  albums  that  can  be  shared  with  other  users  who  are  considered  ‘friends’. 
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This  study  examines  Facebook  usage  and  how  students  use  it  to  engage  in  the  academic  realm,  including 
student  perceptions  of  faculty  as  friends.  As  OSNWs  continue  to  thrive  in  today’s  environment,  faculty  are 
increasingly  implementing  them  in  the  classroom.  Some  feel  this  is  an  invasion  of  the  students*  privacy,  and  have 
labeled  this  as  a ’’creepy  treehouse**  practice  (Abel,  2005;  Stein,  2008).  On  the  other  hand,  others  feel  that  it  is  an 
intelligent  use  of  current  technologies  in  the  classroom.  Since  classroom  community  has  substantial  impact  on  the 
overall  collegiate  experience,  more  needs  to  be  learned  about  this  virtual  community  that  serves  as  an  undercurrent 
communication  channel  that  binds  students  and  faculty.  Coupled  with  student  usage  of  Facebook  (i.e.  hours  spend, 
number  of  friends,  number  of  groups  subscribed  to,  etc.),  is  there  any  relationship  between  faculty  interaction  and 
academic  benchmarks  such  as  cumulative  GPA? 

Literature  Review 

The  popularity  of  OSNWs  such  as  My  Space  and  Facebook  has  grown  considerably  in  recent  years 
(Gosling,  Gaddis  & Vazire,  2007).  However,  any  social  environment  involves  personal  disclosure,  leading  to  issues 
of  privacy  (Acquisti  and  Gross,  2005;  Gemmill  and  Peterson,  2006;  Gross,  Acquisti  & Heinz,  2005;  Kolek  & 
Saunders,  2008;  Patil  & Kosba,  2005;  Tufekci,  2009).  As  the  Pennsylvania  State  example  illustrates,  privacy  issues 
persist  on  OSNWs.  Patil  and  Kobsa  (2005)  studied  the  use  of  privacy  controls  within  awareness  systems,  and 
establish  that  the  use  of  privacy  controls  in  technology  is  dependent  on  the  knowledge  of  security  features,  and  the 
technology  itself.  Users  of  MySpace  were  presented  security  concerns  and  chose  not  to  enable  more  privacy 
settings. 

But  what  do  people  disclose  on  ONSWs?  Gross,  Acquisti  and  Heinz  (2005)  examined  4,540  Facebook 
users  for  the  type  and  amount  of  information  disclosed  and  found  that  an  overwhelming  majority  of  profiles  provide 
full  access,  associating  the  person  with  their  first  and  last  name,  picture,  birthday,  and  hometown.  More  than  half 
provided  their  current  residence.  The  majority  of  users  provide  fully  identifiable  information  although  the  sites  do 
not  require  disclosure.  The  study  concluded  that  few  users  change  the  privacy  settings  and  seem  willing  to  provide 
personal  information  to  the  public.  Kolek  and  Saunders  (2008)  also  found  that  a great  number  of  students  in  their 
quantitative  study  disclosed  substantial  information  on  their  account  profiles  such  as  contact  information,  academic 
schedules  and  personal  pictures  of  alcohol  consumption.  Given  the  low-privacy  settings,  high  personal  disclosure, 
and  college-age  participants  show  very  little  concern  over  privacy  issues  in  ONSWs  (Acquisti  and  Gross,  2005; 
Gemmill  and  Peterson,  2006;  Gross,  Acquisti  & Heinz,  2005;  Kolek  & Saunders,  2008;  Patil  & Kosba,  2005; 
Tufekci,  2009). 

Gemmill  and  Peterson  (2006)  surveyed  student's  technology  use  behavior  and  habits.  Information  was 
gathered  regarding  the  use  of  email,  instant  message  services,  internet  for  academic  and  leisure,  cell  phone  use  and 
land-line  phone  use.  The  study  found  that  college  students  obtain  social  support  via  cell  phones  and  instant  message 
and  the  use  of  technology  is  likely  to  increase  with  the  advent  of  social  networking  technology.  The  study 
concluded  that  technology  use  surveyed  was  highest  among  freshman  and  lowest  in  seniors  and  in  order  to  avoid 
academic  side  effects,  users  need  to  address  the  role  of  technology  within  their  academic  progress. 

The  world  of  ONSWs  also  may  have  ramifications  on  the  student-teacher  relationship  both  in  and  outside 
the  classroom,  given  that  ONSWs  provides  a virtual  realm  in  addition  to  the  pre-existing  physical  one  (Li  & Pitts, 
2009).  Previous  research  provides  some  insight  into  non-academic  social  exchanges  between  college  teachers  and 
students,  finding  that  academic  performance  increases  with  informal  interactions  (Pascarella  and  Terenzini,  2005). 

The  amount  of  time  that  undergraduate  students  invest  in  OSNWs  varies,  but  studies  conclude  that 
Facebook  is  fully  integrated  in  the  lives  of  most  undergraduate  students.  Pempek,  Yermolayeva  and  Calvert  (2009) 
found  that  undergraduate  students  invest  approximately  30  minutes  daily  (27.93  on  weekdays,  28.44  minutes  on 
weekend  days)  on  Facebook , adding  this  task  to  their  daily  routine. 

And  the  time  that  is  invested  may  not  be  devoted  to  academic  enhancement.  In  other  words,  medians  like 
Facebook  and  other  OSNWs  are  used  more  for  socializing  rather  than  academic  usage  (Madge,  Meek,  Wellens  & 
Hooley,  2009).  Madge,  et  al.  (2009)  surveyed  British  undergraduates  and  found  that  undergraduates  were  uneasy 
with  academic  utilitization  of  ONSWs. 

There  were  several  activities  that  the  majority  of  students  did  not  use  Facebook  for,  such  as 
checking  out  the  profile  of  a member  of  university  staff  (68%  said  they  had  never  done  this). 
Facebook  is  therefore  currently  used  by  students  for  communicating  with  other  students,  not  with 
university  staff. . .Moreover,  when  respondents  were  asked  if  there  were  any  ways  they  thought 
Facebook  could  be  utilised  to  enhance  teaching  and  learning  at  the  University,  43%  responded 
negatively,  explaining  that  Facebook  was  a SNS  [Social  Network  Sites],  not  a tool  for  academic 
work  (p.  149). 

So  what  are  the  academic  costs  of  spending  a substantial  amount  of  time  on  ONSWs?  To  date,  research  on  the 
academic  implementation  of  Facebook  or  other  ONSWs  has  been  limited. 
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With  the  seeming  ubiquity  of  Facebook  and  other  ONSWs,  researchers  are  beginning  to  note  the  viability 
of  their  use  in  the  academic  arena.  An  article  in  Educause  (2006)  suggested,  “Any  technology  that  is  able  to 
captivate  so  many  students  for  so  much  time  not  only  carries  implications  for  how  those  students  view  the  world  but 
also  offers  an  opportunity  for  educators  to  understand  the  elements  of  social  networking  that  students  find  so 
compelling  and  to  incorporate  those  elements  into  teaching  and  learning.”  Along  the  same  lines,  Cloete,  de  Villiers, 
and  Roodt  (2009)  found  that  most  faculty  members  have  not  implemented  an  ONSW  as  an  academic  tool,  but  at  the 
same  time,  most  felt  like  one  could  be  applied  as  a tool  for  academic  learning. 

One  of  the  biggest  concerns  in  implementing  Facebook  into  the  classroom  is  student  privacy.  Many  of  the 
studies  about  taking  the  faculty/student  relationship  online  have  focused  on  this  specific  student  concern  (Ellison, 
Steinfield,  & Lampe,  2007;  Hewitt  & Forte,  2006;  Mack,  Behler,  Roberts,  & Rimland,  2007).  Some  students  are 
worried  that  faculty  might  form  opinions  about  them  based  on  their  online  accounts  (Abel,  2005).  However, 
students,  when  presented  with  the  option  did  find  a beneficial  reason  to  use  Facebook  with  faculty.  Mazer,  Murphy, 
and  Simonds  (2007)  found  that  if  faculty  members  have  Facebook  accounts,  students  are  likely  to  base  decisions  on 
whether  or  not  to  take  a class  from  someone  can  be  affected  by  the  amount  of  information  disclosed  online  by  the 
faculty  member. 

Another  area  potentially  affected  by  implementing  faculty  Facebook  usage  in  the  classroom  is 
faculty/student  communication.  Sturgeon  and  Walker  (2009)  found  that  students  seem  to  be  more  willing  to 
communicate  with  their  instructors  if  they  already  knew  them  through  Facebook.  Hewitt  and  Forte  (2006)  similarly 
found  that  students  liked  the  potential  to  get  to  know  their  professors  better,  and  that  Facebook  interaction  had  a 
positive  impact  on  how  they  perceived  their  professors.  Additionally,  Haspels  (2008)  found  that  faculty  Facebook 
usage  also  had  a positive  effect  on  the  face-to-face  faculty/student  relationship. 

But  most  importantly,  does  a faculty/student  relationship  in  Facebook  have  an  effect  on  student 
performance?  Studies  have  found  differing  results.  Yang  and  Tang  (2003)  found  that  those  networks  which  “consist 
of  relations  through  which  individuals  share  resources  such  as  information,  assistance,  and  guidance”  (p.  96)  are 
“positively  related  to  student  performance”  (p.  93)  both  in  face  to  face  and  online  settings.  Sturgeon  and  Walker 
(2009)  found  what  they  termed  an  “indirect  connection  between  faculty  use  of  Facebook  and  academic 
performance”  (p.  11).  Their  findings  postulate  that  because  of  an  increase  in  faculty/student  familiarity,  students  feel 
more  comfortable  and  therefore,  are  able  to  learn  better.  However,  a preliminary  study  by  Karpinski  and  Duberstein 
(2009)  showed  that  students  who  self-reported  spending  more  time  on  Facebook  had  lower  GPAs  than  those  who 
spent  less  time  there. 

Methods 

We  constructed  a survey  instrument  to  ascertain  the  behavior,  attitudes  about  and  Facebook  usage  as  a 
mechanism  to  develop  a sense  of  community.  Regarding  the  voyeuristic  nature  of  Facebook,  we  also  included 
questions  over  account  disclosure  and  privacy  concerns.  Our  45-question  instrument  covered  a variety  of  issues  such 
as  privacy  issues,  faculty  interaction  on  Facebook  and  social  interaction  tendencies  (see  Appendix  A).  In  addition,  a 
series  of  questions  are  included  to  assess  demographics  of  the  students,  which  also  serve  as  independent  variables. 

We  hypothesize  that  students  with  professors  or  instructors  have  a higher  self-reported  GPA  than  those  who 
have  none  listed  as  friends.  In  other  words,  our  hypothesis  is; 

P not  Jxiend  " M prof-friend  ^ 0 

In  addition  to  using  descriptive  statistics,  we  analyzed  the  collected  data  using  T-tests  to  examine  whether 
any  differential  between  those  students  who  had  faculty  as  a Facebook  friend  and  those  who  did  not  was  statistically 
significant.  In  addition  to  collecting  demographic  information  (i.e.,  age,  gender,  ethnic  background,  selected  major, 
classification,  etc.)  and  selected  questions  about  Facebook  usage  and  privacy  issues,  we  asked  the  respondents  how 
many  faculty  members  they  friended  on  Facebook.  We  coded  those  who  had  one  or  more  faculty  members  as 
Facebook  friends  as  “1”  and  those  who  had  none  as  “0”. 

The  institution  used  in  this  study  is  a research  university  in  the  central  US  that  serves  a student  population 
over  20,000  at  its  flagship  campus.  The  campus  houses  approximately  5,900  students:  4,400  of  which  reside  in 
single  student  housing  and  the  remaining  1,400  students  live  in  family  housing.  Our  sample  is  drawn  from  Facebook 
users  residing  in  on-campus  single  student  housing.  A screening  question  regarding  if  the  student  has  a Facebook 
account  was  used  to  screen  out  all  students  who  do  not  have  a Facebook  account.  These  students  were  told  to  skip 
to  the  end  of  the  survey  and  not  answer  any  further  questions. 

Since  this  study  examines  online  behavior  and  interaction,  we  used  Microsoft  SharePoint  to  disseminate  the 
survey  to  residential  life  students  on  December  2008.  Data  collection  continued  for  three  weeks.  In  light  of  the 
recommendations  of  Dillman  (2007),  we  contacted  the  participants  three  times  throughout  the  data  collection 
process.  The  first  contact  will  occurred  at  the  invitation  to  participate  in  the  study,  and  reminder  emails  were 
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disseminated  a week  apart  afterwards.  There  were  no  incentives  given  to  participate  in  the  study.  We  received  a 13 
percent  response  rate  to  our  online  survey  that  was  disseminated  in  the  Fall  of  2008  to  5,013  residential  life  students. 

Descriptive  Statistics 

Of  the  respondents,  49  percent  self-reported  themselves  as  freshman,  22  percent  as  sophomores,  15  percent 
were  juniors,  9 percent  as  seniors  and  8 percent  were  graduate  students.  Females  constituted  the  majority  of  the 
respondents,  64  percent  while  36  percent  were  males.  When  looking  at  ethic  backgrounds,  77  percent  of  the 
participants  were  White,  7 percent  were  American  Indians,  6 percent  were  Hispanic,  6 percent  were  Asian  and  4 
percent  was  were  of  African  American  descent. 

In  regards  to  community,  first,  we  found  that  72  percent  of  the  respondents  were  active  in  some  student 
organization  while  28  percent  reported  no  affiliation.  Of  the  respondents,  86  students  belonged  to  a Greek 
organization  while  485  students  did  not.  Within  the  organization  community,  students  are  using  Facebook  groups  as 
a viable  means  of  communication.  Forty-two  percent  of  respondents  use  Facebook  groups  to  contact  organization 
members,  41  percent  by  email  while  1 1 percent  use  phone.  The  popularity  of  email  among  respondents  also  aligns 
with  recent  findings  (Boase,  et  ah,  2006,  Boase  & Wellman  & Gulia,  2006;  Haythornwaite,  2001;  Stern  & Dillman, 
2006),  but  Facebook  is  at  the  very  least  on  par  with  e-mail  as  a communication  option  among  students. 

As  for  social  community  participation,  a sizeable  majority  of  survey  respondents,  84.4  percent,  reported  to 
have  more  than  100  Facebook  friends  while  66.4  percent  disclosed  that  they  have  more  than  200  Facebook  friends. 
Clearly,  Facebook  is  an  avenue  for  social  community  development.  At  the  same  time,  Facebook  usage  is  not  merely 
social.  It  is  also  being  used  to  support  an  academic  community. 

Thirty  percent  of  respondents  reported  having  a faculty  member  as  a friend  on  Facebook.  We  also  asked 
about  the  effect  of  Facebook  on  professor  selection.  While  56  percent  of  respondents  reported  that  they  would  be 
more  apt  to  take  a professor  if  they  liked  his/her  profile,  53  percent  reported  that  they  would  be  more  apt  not  to  take 
a professor  if  they  disliked  his/her  profile.  Within  the  academic  community,  88  percent  of  respondents  used  email  as 
a primary  means  to  contact  their  instructor  and  12  percent  use  office  hours  as  a secondary  method.  This  is  not 
surprising  when  considering  email  communication  increases  as  one’s  social  ties  increase  and  email’s  asynchronous 
and  convenient  nature  (Boase,  et  ah,  2006).  For  the  professor,  his/her  own  social  ties  increase  according  to  the 
number  of  students  in  a given  class.  When  asked  about  the  secondary  method  of  contacting  their  instructor  or 
professor,  78  percent  of  respondents  preferred  office  hours,  14  percent  listed  email,  6 percent  voicemail  and  2 
percent  listed  Facebook  as  a means  of  communication. 

The  survey  instrument  also  inquired  about  issues  of  privacy  and  disclosure  that  painted  a very  interesting 
picture  on  Facebook  usage.  When  asked  whether  their  Facebook  profile  privacy,  74  percent  of  respondents 
disclosed  that  their  profiles  were  private  while  26  admitted  that  their  profile  was  open.  In  addition,  70  percent 
reported  that  they  post  no  residence  information  on  their  Facebook  profile  while  22  percent  list  their  residence  halls 
and  9 percent  post  both  their  residence  hall  information  and  room  number.  Respondents  were  less  restrictive  on 
access  to  their  photos.  Surprisingly,  55  percent  of  respondents  disclosed  that  they  make  their  photos  available  to 
their  entire  network  while  45  percent  did  not.  However,  most  of  the  respondents  have  taken  action  to  restrict  access 
to  their  profiles.  When  asked  whether  they  have  blocked  anyone  (i.e.,  blocked  their  profile  access  to  a particular 
person  or  people)  on  Facebook,  47  percent  responded  affirmatively  while  53  percent  have  not.  Sixty  percent  of 
survey  respondents  repotted  that  they  have  limited  access  to  certain  individuals  to  their  profile  (i.e.,  allowed  general 
access,  but  limited  access  to  photos,  information,  etc.)  while  40  percent  have  not.  And  lastly  and  most  disturbing,  a 
large  percentage  of  respondents  (47  percent)  ha  s/e.  friended  people  on  their  Facebook  profile  who  they  do  not  know. 

Analysis 

When  analyzing  the  data,  we  found  that  those  students  who  had  professors  as  Facebook  friends  had  a 
higher  self-reported  GPA  than  those  who  did  not.  Those  with  professor  or  instructors  as  friends  as  a GPA  of  3.42  as 
compared  to  those  who  did  not  with  a GPA  of  3.33. 

Table  1:  Descriptive  Statistics 


N 

Mean 

cumulative  GPA 

Standard 

deviation 

Standard  Error 

Professor  Not 
Friend 

349 

3.33 

.58 

.03 

Professor  as 
Friend 

179 

3.42 

.51 

.39 
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Although  outside  of  the  purview  of  this  study,  we  additionally  tested  various  variables  to  see  whether  there 
was  a correlation  with  their  online  activities  (i.e.,  number  of  self-reported  hours  spent  on  Facebook,  number  of  self- 
reported  Facebook  friends  and  number  of  self-reported  Facebook  groups  joined)  and  their  self-reported  GPA. 
However,  after  conducting  T-tests,  none  of  the  relationships  had  any  statistical  significance.  However,  when  testing 
our  hypothesis,  we  found  that  the  difference  of  self-reported  GPA  between  those  students  who  had  friended  an 
instructor  (GPA  = 3.42)  compared  to  those  who  did  not  (GPA  = 3.33)  was  statistically  significant  at  the  .05  level. 


Table  2:  T-test  analysis  for  cumulative  GPA  between  Two  groups  of  students 


Variance 

T 

Df 

Significance 

Unequal 

1.965 

387 

.05 

Therefore,  our  analysis  reveals  that  those  students  who  have  friended  one  or  more  faculty  members  are 
more  likely  to  have  a higher  self-reported  GPA  than  those  who  have  no  faculty  members  listed  as  friends  on 
Facebook.  In  the  next  section  of  our  paper,  we  will  discuss  the  significances  of  this  finding  and  the  results  of  our 
descriptive  statistics. 

Discussion 

The  discussion  of  our  results  is  organized  into  three  different  sections  that  paints  a rich  picture  of  Facebook 
student  usage,  disclosure,  privacy  and  faculty/staff  interaction.  First,  we  discuss  the  descriptive  findings  that  involve 
privacy  issues  on  Facebook , discussing  some  key  issues.  Second,  the  findings  involving  faculty  interaction  are  also 
discussed  through  descriptive  statistics  and  the  ANOVA  model  of  our  student.  Lastly,  we  provide  policy 
implications  and  a conclusion  at  the  end  of  this  paper. 

Facebook  Usage  and  Privacy 

Two  observations  stand  out  after  examining  the  survey  results,  both  having  a relationship  with  Facebook 
usage  and  privacy  concerns.  First,  in  terms  of  privacy,  there  remain  a large  percentage  of  students  who  maintain 
their  profile  to  be  open  to  the  public  at  26  percent.  Although  some  studies  have  lamented  over  the  reasons  for  this 
disclosure  needs,  other  studies  have  concluded  that  students  simply  do  little  to  protect  their  own  disclosures  in 
ONSWs  (Acquisti  and  Gross,  2005;  Gemmill  and  Peterson,  2006;  Gross,  Acquisti  & Heinz,  2005;  Kolek  & 
Saunders,  2008;  Patil  & Kosba,  2005;  Tufekci,  2009). 

However,  the  survey  results  may  deviate  from  the  literature.  Selywn  (2009)  found  that  Facebook  serves  as 
a supplement  to  pre-existing  relationships  and  student  typically  do  not  use  it  to  make  friends  who  they  may  not  have 
met  in  person.  However,  the  results  of  our  survey  are  mixed.  Nearly  half  of  those  surveyed  conceded  that  they  have 
Facebook  friends  who  they  have  never  met  which  conflicts  with  Selywn  article  and  raises  serious  safety  concerns. 
However,  they  generally  do  not  use  Facebook  to  make  personal  connections  on  a college  campus  before  enrolling. 
However,  when  asked  whether  those  surveyed  sought  Facebook  friends  at  the  university  before  they  arrived  on 
campus,  only  34  percent  admitted  so.  One  explanation  could  be  that  students  may  accumulate  unmeet  Facebook 
friends  through  other  friends,  but  not  purposely  to  pre-establish  a social  network  on  campus  prior  to  arriving. 

Faculty/student  interaction 

With  an  OSNW  that  relies  on  college  populations  as  consumers,  it  is  not  surprising  that  there  is  faculty  and 
student  interaction  on  Facebook.  The  connection  between  faculty  and  student  interaction  appears  to  have  positive 
benefits.  First,  students  could  benefit  from  the  interaction  that  comes  with  communication  on  Facebook.  Research 
has  shown  that  general  interaction  between  faculty  and  students  increase  academic  performance  (Kuh,  Cruce, 

Shoup,  Kinzie  & Gonyea,  2008;  Carini,  Kuh  & Klein,  2006;  Pascarella  & Terenzini,  2005;  Kuh,  2001).  While  most 
of  the  literature  examines  general  faculty  and  student  interaction  (Carini,  Kuh  & Klein,  2006;  Kuh,  2001),  there  is 
some  evidence  that  suggests  even  informal  student  interaction  provides  students  with  some  positive  benefit  whether 
it  be  higher  student  satisfaction  (Kuh  & Hu,  2001)  or  higher  educational  aspirations  or  even  academic  performance 
(Pascarella,  1980;  Pascarella  & Terenzini,  2005).  Perhaps  the  same  can  be  said  about  interaction  on  OSNWs. 

Second  and  more  cynically,  professors  and  instructors  may  be  more  apt  to  friend  academically  talented  or  students 
who  perform  better  in  their  class  as  oppose  to  students  who  do  poorly  in  class.  On  the  other  hand,  higher  performing 
students  may  feel  more  comfortable  with  friending  a faculty  member  than  low-performing  students.  To  further 
investigate  these  ONSW  interactions,  future  research  can  examine  as  to  who  extends  friend  invitations  and  why 
faculty  friend  some  students  and  not  others. 
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Implications 

However,  our  findings  do  not  suggest  that  faculty  and  staff  should  use  Facebook  as  a mechanism  for 
enhancing  academic  performance  or  student  satisfaction.  The  literature  and  the  descriptive  statistics  from  this  survey 
recommend  otherwise.  First,  when  contacting  their  instructors  or  professors,  students  opt  to  communicate  through 
email  then  office  hours.  Facebook  and  voice  mail  are  distant  thirds  and  fourths.  In  addition,  only  a minority  of  30 
percent  has  a professor  or  instructor  listed  as  a friend.  The  literature  aligns  with  this  finding  and  suggests  that 
students  see  OSNWs  more  for  socializing  rather  than  for  academic  usage  (Connell  2009;  Madge,  Meek,  Wellens  & 
Hooley,  2009).  In  examining  whether  Facebook  would  be  proper  median  for  outreach,  Connell  (2009)  found  that  a 
good  number  of  students  were  not  favorable  to  be  friended  by  the  library,  possibly  seeing  it  as  an  intrusion  into  their 
realm. 

Some  participants  (63  or  17.2  percent)  were  very  open  to  the  idea  and  said  that  they  would  be 
proactive  and  invite  the  library  to  be  their  friends  if  they  know  about  the  account.  The  majority  of 
respondents  (211  or  57.5  percent)  said  that  they  would  not  be  proactive  about  it,  but  if  the  library 
friended  them,  they  would  accept  the  friend  invitation.  Another  group  of  92  students  (25.1  percent) 
said  that  they  would  not  add  the  library  as  a friend,  (p.  31). 

This  study  only  illustrates  that  a relationship  does  exist  between  self-reported  GPA  and  Facebook  friend 
status  with  a student.  There  is  certainly  no  evidence  to  suggest  that  faculty  should  use  their  Facebook  account  for 
improved  academic  performance.  Such  a proactive  approach  would  likely  be  met  by  disappointing  results  by 
encroaching  on  perceived  social  space.  Instead,  if  faculty  were  inclined  to  open  their  Facebook  account  to  students, 
promoting  its  availability  as  a median  of  communication  may  be  a wiser  choice  rather  than  sending  invitations  to 
enroll.  When  examining  library  outreach  efforts  through  Facebook,  Connell  (2009)  warned: 

Therefore,  perhaps  indiscriminate  friending  is  not  a good  idea.  It  is  important  not  to  annoy 
students  but  rather  let  them  come  to  the  library  on  their  own  terms  (p.  34). 

In  all,  we  must  be  cognizant  that  Facebook  is  in  fact  the  realm  of  the  digital  native,  and  students  may  view 
any  unilateral  outreach  by  faculty  or  staff  as  encroachment  on  their  turf  and  instigate  a ‘creepy  treehouse’  effect. 

Conclusion 

Student  usage  of  OSNWs  is  nothing  short  is  prolific  and  information  from  the  literature,  as  well  as 
anecdotal  evidence,  shows  that  this  trend  will  most  certainly  continue.  Students  will  continue  to  utilize  web-based 
social  networking  as  an  online  communication  fomm,  but  mostly  for  informal,  social  interaction  with  other  students. 
Those  few  students  who  have  professors  or  instructors  as  friends  have  been  found  to  have  higher  self-reported  GPAs 
compared  to  those  who  don't.  Although  this  relationship  was  found,  much  more  can  be  learned  about  OSNWs, 
college  life  and  the  faculty-student  relationship.  By  conducting  this  study,  the  authors  sought  to  shed  more  light  in 
how  OSNWs  intercross  with  the  academic  realm. 
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1 .  Do  you  live  on  campus? 

a.  Yes 

b.  No 


Appendix  A 


2.  Estimate  your  family  income: 

a.  $20,000  or  less 

b.  $20,001  to  $50,000 

c.  $50,001  to  $100,000 

d.  $100,001  or  more 

3.  What  types  of  financial  aid  do  you  receive?  (pick  all  that  apply): 

a.  Stafford 

b.  Pell 

c.  OHLAP 

d.  Private  loans 

e.  Other  state  grant 

f.  Etc. 

4.  What  best  describes  your  ethnic/racial  background? 

a.  American  Indian  or  Alaska  Native 

b.  Asian 

c.  Black  or  African  American 

d.  Native  Hawaiian  or  Other  Pacific  Islander 

e.  White 

f.  Hispanic  or  Latino 

5.  Which  of  the  following  online  social  network  websites  do  you  use  the  most? 

a.  Facebook 

b.  MySpace 

c.  Friendster 

d.  Ball  of  Dirt 

e.  Other 

6.  What  is  your  gender? 

a.  Male 

b.  Female 

7.  What  is  your  cumulative  GPA  in  college  on  a 4.0  scale? 

8.  Are  you  a member  of  Greek  organization? 

a.  Yes 

b.  No 

9.  Do  you  live  in  Greek  housing? 

a.  Yes 

b.  No 

10.  Are  you  a member  of  a non-Greek  student  organization? 

a.  Yes 

b.  No 
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11.  What  is  your  age? 

a.  17-18 

b.  19-20 

c.  21-25 

d.  26  or  older 

12.  What  best  describes  your  major? 

a.  Education 

b.  Humanities 

c.  Engineering 

d.  Social  science 

e.  Health/medical 

f.  Other 

13.  What  is  your  classification? 

a.  Freshman 

b.  Sophomore 

c.  Junior 

d.  Senior 

e.  Graduate  student 

14.  What  best  describes  your  political  orientation 

a.  Very  conservative 

b.  Conservative 

c.  Moderate 

d.  Liberal 

e.  Very  Liberal 

f.  Apathetic 

15.  What  is  your  primary  method  of  contacting  your  instructor  or  professor? 

a.  By  email 

b.  By  voicemail 

c.  By  Facebook 

d.  By  office  hours 

16.  What  is  your  secondary  method  of  contacting  your  instructor  or  professor? 

a.  By  email 

b.  By  voicemail 

c.  By  Facebook 

d.  By  office  hours 

17.  How  do  you  communicate  with  members  of  student  organization  to  which  you  belong? 

a.  By  email 

b.  By  Facebook  groups 

c.  By  phone 

d.  By  flyer 

e.  By  Other  means 

18.  If  you  have  a Facebook  account,  how  many  friends  have  you  added  on  Facebook?  
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19.  What  is  your  primary  method  of  contacting  your  resident  assistant? 

a.  By  email 

b.  By  voicemail 

c.  By  Facebook 

d.  By  office  hours 

e.  By  instant  messaging 

f.  By  text  messaging 

20.  What  is  your  secondary  method  of  contacting  your  resident  assistant? 

a.  By  email 

b.  By  voicemail 

c.  By  Facebook 

d.  By  office  hours 

e.  By  instant  messaging 

f.  By  text  messaging 


Please  answer  the  following  questions  if  you  have  a Facebook  account 

21.  What  residence  information  do  you  post  on  Facebook? 

a.  I post  no  information 

b.  Residence  hall 

c.  Residence  hall  and  room  number 


22.  Do  you  make  your  photos  public  to  everyone  in  your  network? 

a.  Yes 

b.  No 


23.  How  many  professors  or  instructors  do  you  have  listed  as  a friend? 

24.  Do  you  have  your  resident  assistant,  community  facilitator  or  community  mentor  listed  as  a friend? 

a.  Yes 

b.  No 


25.  Which  of  these  do  you  feel  comfortable  posting  on  Facebook  (if  you  have  open  access  to  your  pictures)? 

a.  Pictures  of  yourself 

b.  Pictures  of  parties 

c.  Pictures  of  family 

d.  Pictures  of  friends 

e.  Pictures  of  vacation 

f.  No  open  access 

26.  Is  your  Facebook  profile  private? 

a.  Yes 

b.  No 


27.  Have  you  created  a group? 

a.  Yes 

b.  No 


28.  How  many  Facebook  groups  are  you  a member  of? 
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29.  Mark  as  many  as  apply:  how  active  are  you  in  these  groups  on  Facebook? 

a.  Read  messages  only 

b.  Read  and  post  messages 

c.  Look  at  pictures 

d.  Post  pictures 

e.  An  administrator  of  a group 

30.  Is  your  mini-feed  public  or  private? 

a.  Private 

b.  Public 

3 1 . Have  you  blocked  anybody? 

a.  Yes 

b.  No 

32.  Have  you  limited  anybody’s  access  of  your  profile  on  Facebook? 

a.  Yes 

b.  No 

33.  How  much  do  you  agree  with  the  following  statement:  Students  would  be  more  academically  engaged  if 
their  professors  were  on  Facebook? 

a.  Strongly  agree 

b.  Somewhat  agree 

c.  Somewhat  disagree 

d.  Strongly  disagree 

34.  How  many  times  do  you  log  into  Facebook  per  week  on  average? 

35.  How  many  hours  do  you  spend  on  Facebook  per  week? 

36.  What  best  describes  your  motivation  for  participating  on  Facebook  (check  all  that  apply): 

a.  Making  and  maintaining  college  friendships 

b.  Maintaining  high  school  friendships 

c.  Dating 

d.  Random 

37.  Do  you  have  people  listed  as  friends  that  you  don't  know? 

a.  Yes 

b.  No 

38.  Would  you  be  more  apt  to  take  a professor  if  you  saw  their  Facebook  profile? 

a.  Yes 

b.  No 

39.  Would  you  be  more  apt  not  to  take  a professor  if  you  disliked  their  Facebook  profile? 

a.  Yes 

b.  No 

40.  Would  you  be  more  apt  to  take  a professor  if  you  liked  their  Facebook  profile? 

a.  Yes 

b.  No 


105 


41.  Do  you  agree  or  disagree  with  the  following  statement:  It  is  appropriate  for  a staff  member  to  use  Facebook 
to  check  on  the  well-being  of  a student. 

a.  Strongly  agree 

b.  Somewhat  agree 

c.  Somewhat  disagree 

d.  Strongly  disagree 

42.  Do  you  agree  or  disagree  with  the  following  statement:  It  is  appropriate  for  a staff  member  to  use  Facebook 
for  university  policy  reinforcement. 

a.  Strongly  agree 

b.  Somewhat  agree 

c.  Somewhat  disagree 

d.  Strongly  disagree 

43.  Do  you  agree  or  disagree  with  the  following  statement:  I feel  connected  to  campus  through  Facebook. 

a.  Strongly  agree 

b.  Somewhat  agree 

c.  Somewhat  disagree 

d.  Strongly  disagree 

44.  Do  you  agree  or  disagree  with  the  following  statement:  I feel  that  there  is  a genuine  campus  community  on 
Facebook. 

a.  Strongly  agree 

b.  Somewhat  agree 

c.  Somewhat  disagree 

d.  Strongly  disagree 

45.  Did  you  seek  Facebook  friends  at  the  university  before  you  arrived  on  campus  as  a student? 

a.  Yes 

b.  No 
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Introduction 

Faculty  at  a Master  degree  granting  institution  in  the  Southern  United  States  are  in  the  process  of  creating  systemic 
and  collaborative  change  as  they  revise  and  expand  an  existing  Educational  Technology  program  that  awards  the 
Master  of  Education  degree  (M.Ed.).  In  addition  to  typical  efforts  required  to  generate  change  within  the  curricular 
system,  the  faculty  have  initiated  intra-departmental  collaboration  to  reduce  curricular  redundancies,  better  sequence 
courses,  align  key  assessments,  and  appropriately  respond  to  diverse  student  needs  and  institutional  expectations  for 
rigor.  This  paper  will  explain  the  background,  impetus  for  change,  the  participants  and  processes  involved  in 
change  including  survey  results,  curricular  outcomes  in  relation  to  course  development  and  outreach,  and 
conclusions. 

Background 

James  Madison  University  was  established  in  early  1908  as  the  State  Normal  and  Industrial  School  for  Women.  A 
teacher's  school,  the  institution  has  traditionally  focused  heavily  on  quality  teaching  and  learning.  It  has  been 
recognized  annually  as  a top.  Master  degree  granting  institution  in  the  South.  Since  its  inception,  the  Education  arm 
of  JMU  has  been  transformed  through  changes  to  name  and  organizational  structure.  In  2003,  the  School  of 
Education  became  the  College  of  Education  and  departments  emerged  from  existing  programs. 

When  first  conceived,  the  Educational  Technology  program  at  James  Madison  University  was  much  like  those  of 
other  Colleges  of  Education.  In  the  1990's,  'technology'  at  the  K12  level  was  an  outgrowth  of  library  programs.  The 
publication  of  Information  Power,  1st  Ed,  in  1998  ( AALA)  led  the  charge  for  'libraries'  to  become  ’media  centers' 
and  for  those  responsible  for  library  programs  to  view  their  role  in  K12  as  one  that  also  includes  the  selection  and 
utilization  of  technology  in  the  instructional  process.  Flexible  scheduling  of  the  media  center  was  promoted  as  a way 
to  include  authentic  use  of  and  provide  instruction  in  the  integration  of  media.  Following  suit,  JMU's  Educational 
Technology  graduate  program  began  with  a Library /Media  focus.  It  was  a media  program  designed  for  those  who 
would  be  teaching  such  media  skills  in  the  K12  environment,  and  in  fact,  was  arranged  around  the  requirements  of 
the  Virginia  Department  of  Education  licensure  requirements  for  school  media  specialists. 

Educational  Technology  is  currently  housed  in  the  Learning,  Technology  and  Leadership  Education  (LTLE) 
department,  established  in  2006.  Other  LTLE  programs  include  a collaborative  doctoral  program  in  Strategic 
Leadership,  and  Masters  programs  in  Educational  Leadership,  and  Adult  Education  and  Human  Resource 
Development  (AHRD).  At  the  undergraduate  level  the  department  offers  Foundations  courses  for  the  College,  a 
minor  in  Educational  Media,  and  a minor  in  Human  Resource  Development.  The  establishment  of  this  department  is 
a reflection  of  the  changing  and  broadening  focus  of  many  Colleges  of  Education  - that  is  to  be  inclusive  of  all 
contexts  of  educational  needs,  not  only  those  in  the  K12  realm.  Specifically,  AHRD  and  Educational  Technology 
recognize  corporate,  non-profit,  and  K12  education  as  potential  contexts  for  the  practice  of  instructional  design, 
development,  delivery,  and  management. 

The  JMU  College  of  Education  is  still  very  young  and  opportunities  to  influence  future  direction  abound.  Each 
College  of  Education  academic  program  must  bring  its  strengths  to  bear  on  the  institutional  change  process.  The 
applied  and  diverse  nature  of  the  JMU  Educational  Technology  Masters  program  (M.  Ed.)  is  a strength  that  supports 
the  goals  and  standards  used  by  a range  of  programs  in  the  College  of  Education.  The  program  is  distinguished  in 
the  College  by  its  learners  who  hale  from  a number  of  academic  disciplines  and  who  bring  with  them  diversity  in 
professional  experience  and  career  goals. 


Impetus  for  Change 


Evolving  Interests 

While  the  Educational  Technology  program  served  the  JMU  College  of  Education  well  for  many  years,  the  early 
21st  century  brought  with  it  an  increased  demand  for  teacher  competency  with  technology  - skills  and  knowledge 
that  were  previously  designated  for  the  media  specialist.  Along  with  this  changing  K12  focus  also  came  the 
recognition  that  the  term  'education'  applies  to  many  contexts,  not  just  that  of  early  education.  The  target  audience 
for  enrollment  was  broadened  to  include  anyone  who  had  previously  worked  in  K12.  School-work  connections  are. 
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and  continue  to  be,  emphasized  in  the  program.  Learning  is  presently  viewed  as  'life-long'  and,  as  such,  the  focus  of 
education  programs  is  no  longer  kindergarten  through  high  school  only.  There  is  now  a strong  recognition  that 
learning  and  development  continue  beyond  the  school  years,  into  adulthood.  It  is  only  natural  then,  that  the  Adult 
Education  and  Human  Resource  Development  program  and  the  Educational  Technology  program  came  to  model 
this  expanded  view  of  education. 

Program  Decline 

Bowling  (2001)  offers  a life -cycle  model  of  declining  programs  that  apply  to  this  scenario.  Characteristics  of 
declining  programs  in  the  model  include:  (1)  a declining  number  of  program  participants;  (2)  falling  evaluation 
ratings;  and  (3)  increasing  interest  in  new  subject  material  beyond  the  current  program  curriculum.  Program 
enrollment  followed  a steady  decline  with  time  and  there  was  little  branding  or  marketing  of  the  graduate  program 
due  to  limited  faculty  resources  and  very  specific  admittance  criteria.  Students  who  lacked  K12  experience  were  not 
admitted  and  instead  tended  to  pursue  a degree  in  Adult  Education  and  Human  Resource  Development.  In  that  way, 
students  could  develop  instructional  design  and  technology  skills  that  were  transferable  to  a corporate  or  non-profit 
setting.  Students  who  graduated  from  the  AHRD  program  often  assumed  roles  as  instructional  designers,  trainers 
and  course  developers. 

In  a 2007  internal  review  of  the  Educational  Technology  program,  it  was  noted  that  more  consistency  was  needed  in 
the  alignment  of  course  objectives  to  assessments.  Ultimately  this  practice  would  ensure  the  alignment  to  national 
standards.  It  is  important  that  applied  programs  undergo  relatively  constant  review  and  revision.  This  challenge  to 
the  field  of  Educational  Technology  tends  to  spur  debates  over  the  nature,  mission,  and  role  of  educational 
technology  in  the  education  arena.  To  address  program  decline  and  help  situate  the  program  in  the  College  of 
Education,  a committee  was  formed  of  faculty  who  specialize  in  K12  and  corporate  Educational  Technology,  Adult 
Education  and  Human  Resource  Development,  and  general  education.  It  is  the  goal  of  this  newly  formed 
educational  technology  committee  to  revise  the  program  to  be  more  holistic  and  reflective  of  lifelong  learning  in 
modern  society.  Each  course  is  scrutinized  for  its  relevance  in  the  program  of  study.  Intellectual  content,  objectives 
and  assessments  are  being  redesigned  to  ensure  alignment  with  the  Association  for  Educational  Communications 
and  Technology  (AECT)  standards. 

Educational  Technology  is  defined  by  the  Association  for  Educational  Communications  and  Technology  (AECT)  as, 
“the  study  and  ethical  practice  of  facilitating  learning  and  improving  performance  by  creating,  using,  and  managing 
appropriate  technological  processes  and  resources”  (p.  1).  By  definition  the  field  can  serve  a number  of  disciplines. 
The  program  curriculum  needed  to  grow  to  better  serve  this  definition  and  to  address  expanding  departmental 
interests. 

Increasing  Committment 

Prior  to  2008,  the  number  and  diversity  of  students  enrolled  in  the  Educational  Technology  program  was  limited.  A 
small  number  of  faculty  managed  to  sustain  the  program  while  maintaining  other  job  responsibilities.  In  2008  the 
Learning,  Technology  and  Leadership  Education  department  hired  two  tenure-track  faculty  to  enhance  the  program. 
One  faculty  specialized  in  K12  and  the  other  in  postsecondary  and  adult  education.  The  hiring  of  the  new  faculty 
members  enabled  the  department  to  broaden  the  program's  focus  to  encompass  learner  interests  across  the  lifespan. 
This  change  demonstrated  a significant  commitment  to  bolstering  a program  with  much  potential. 

Curricular  Redundancies 

An  initial  comparison  of  intra-departmental  programs  revealed  curricular  consistencies  between  Educational 
Technology  and  Adult  Education  and  Human  Resource  Development,  where  students  pursue  careers  in  training  and 
organizational  change  - largely  in  response  to  innovation.  Evidence  of  curricular  overlap  has  been  identified  in  the 
professional  core,  in  the  program  core,  in  advanced  areas  of  concentration,  and  in  key  assessments.  The  curricular 
overlap  is  due,  in  part,  to  the  increased  requirement  for  technology  proficiency  among  training  professionals  in  the 
corporate  sector,  and  for  the  same  competencies  in  technology  educators.  In  other  words,  the  courses  in  both 
programs  have  addressed  the  needs  of  the  marketplace  by  adapting  the  program  to  include  greater  coverage  of 
instructional  design  and  technology  use.  As  shown  in  Table  1,  the  two  programs  share  four  core  courses  and  four 
key  assessments.  In  addition,  the  faculty  have  proposed  the  addition  of  Educational  Technology  Management  to  the 
AHRD  core  to  address  the  need  for  competence  in  strategic  project  and  technology  management  among  students 
preparing  to  lead  training  organizations. 
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Table  1.  Overlapping  Professional  Core  and  Program  Core  Courses  and  Key  Assessments 


Educational  Technology 

Adult  and  Human  Resource 
Development 

Shared  Key  Assessments 

Foundations  of  Educational 
Technology 

Foundations  of  Adult  Education  and 
Human  Resource  Development 

Inquiry  in  Education 

Inquiry  in  Education 

Curriculum  Theory 

Needs  Assessment  and  Performance 
Analysis 

Instructional  Design 

Instructional  Design 

Design  and  Development  of  Digital 
Media 

Design  and  Development  of  Digital 
Media 

Preliminary  Portfolio  Review 

Learning  Theories 

Learning  Theories 

Educational  Technology  Management 
(proposed  also  for  AHRD  students) 

Leadership  and  Facilitation 

Developing  and  Critiquing  Visual 
Literacy 

Program  Evaluation 

Qualifying  Exam 

Advanced  Coursework 

Advanced  Coursework 

Advanced  Coursework 

Applied  research/inquiry  project; 
final  portfolio  submitted  at  program 
conclusion 

Overlaps  in  advanced  coursework  occurred  because  of  evolving  interest  in  eLearning  and  the  specific  challenges 
that  such  an  approach  brings  to  instructional  designers  and  trainers.  It  is  becoming  increasingly  difficult  to  discuss 
and  practice  instructional  design  removed  from  the  context  of  rich  technological  learning  environments,  and  in  fact, 
may  now  be  impossible. 

Demps  (2008)  identified  consistencies  in  Human  Resource  Development  and  Educational  Technology  programs  and 
suggested  that  gaps  in  one  field  could  be  filled  by  the  theories  and  practices  of  the  other.  Utilization  is  defined  as  the 
"act  of  using  processes  and  resources  for  learning"  (AECT  p.  141)  and  is  directly  related  to  training  and  development 
that  is  typically  found  within  the  HRD  field.  Likewise,  human  performance  and  development  is  of  foundational 
interest  to  the  HRD  field  and  must  use  current  technologies  in  order  to  exist.  Educational  technologists  bring  to  the 
table  the  application  of  eLearning  strategies,  technologies  and  management  to  core  HRD  contexts.  Collaboration 
between  the  programs  supports  a departmental  move  to  expand  the  existing  focus  on  K12  careers  in  Educational 
Technology  to  include  learners  with  non-K12  career  interests.  The  strengths  of  each  program  are  used  as  a 
springboard  for  "renewal,  growth,  and  transformation"  (2009  AECT  call  for  proposals). 

An  unintended  outcome  in  the  various  collaborations  was  that  the  Learning,  Technology,  and  Leadership  Education 
(LTLE)  department  was  taking  on  a branding  and  presence  of  its  own  within  the  College.  College  of  Education 
programs  relied  on  LTLE  to  provide  foundational  courses  in  diversity,  technology,  and  educational  leadership.  The 
overlap  of  core  courses  in  the  AHRD  and  Educational  Technology  Master's  programs  prompted  a departmental 
discussion  about,  and  subsequent  introduction  of,  the  LTLE  course  prefix  to  validate  the  combined  strength  of  these 
expanded  educational  programs  with  the  foundational  courses  provided  to  the  larger  University. 

Change  Participants  and  Processes 

The  basic  core  competencies  involving  theory,  research,  and  design  in  College  of  Education  M.  Ed.  programs  are 
determined  by  the  College  and  department,  with  national  standards  in  mind.  Change  to  the  program  core  and 
advanced  courses  and  certificates  are  influenced  by  national  standards,  the  department,  program,  and  input  from  the 
larger  community.  In  addition,  a systemic  process  for  change  requires  looking  at  the  process  as  a whole  and  not  just 
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a series  of  parts.  Change  also  consists  of  multiple  dimensions  that  must  be  understood  and  appying  a systems 
strategy  to  change  is  required  in  order  to  integrate  the  parts;  select  and  coordinate  types  of  changes;  involve 
stakeholders  and  their  needs;  and  ensure  that  the  end  result  is  viable  within  the  context  of  its  surrounding  system 
(Ellsworth,  2000.)  Likewise,  the  people  who  will  implement  the  innovation  must  have  sufficient  knowledge  to  do 
so.  While  JMU's  Educational  Technology  faculty  were  certainly  qualified  in  their  field  and  able  to  determine  broad 
program  goals  and  needs,  it  was  also  important  to  survey  the  feelings  and  needs  of  those  affected  by  the  program 
and  changes  - more  specifically  those  in  K12  and  corporate  educational  technology  and  related  fields.  Such  a survey 
of  the  field  would  provide  feedback  about  what  happens  when  those  trained  in  Educational  Technology  programs 
attempt  to  operationalize  their  knowledge  and  skills.  An  informal  survey  of  those  involved  in  educational 
technology  in  and  out  of  K12  was  designed  by  the  JMU  Educational  Technology  program  faculty  and  distributed 
locally,  nationally,  and  internationally  to  the  ITFORUM  listserv.  (Appendix  A) 

Survey  participants  included  the  K12  positions  of  Director  of  Technology  (3);  Instructional  Technology  Resource 
Teacher  (3);  and  Supervisor  of  Instructional  Technology  ( 1).  Non-K12  respondants  held  the  following  positions: 
Learning  Technology  Manager;  Instructional  Designer  (3);  Independent  eLearning  Consultant;  Director-Learning 
Resources  and  Instructional  Technology  Centers;  Coordinator  for  Special  Technology  Projects;  and  Distance 
Learning  Coordinator.  Responses  provided  confirmation  of  the  reorganization  of  JMU's  Educational  Technology 
program  to  be  inclusive  - that  the  skills  and  knowledge  are  not  really  context  specific  - but  an  effective  educational 
technologist  must  have  a certain  core  of  knowledge  and  skills  that  can  be  applied  to  any  educational  context,  in 
short,  a generalist.  The  AECT  standards  provide  such  a guidepost  for  program  and  course  development.  Not 
surprisingly,  the  question  regarding  application  programs  used  brought  a gamut  of  responses  - from  free  resources 
such  as  Google  Apps  to  more  expensive,  specialized  content  creation  applications  such  as  Adobe  Photoshop, 
Dreamweaver,  Flash,  and  other  high-end  development  programs.  (Appendix  B)  Loud  and  resounding  themes  were 
the  need  for  more  educational  technology  management  skills,  project  management,  resource  management,  and 
evaluation  skills.  This  input  influenced  the  committee's  thinking  about  advanced  courses  and  certificates  designed 
for  the  Educational  Technology  M.  Ed.  program. 

These  courses  must  meet  the  needs  of  a current  and  unpredictable  future  market  demand.  New  technologies  emerge 
every  day  and  their  influence  on  our  daily  lives  and  the  way  we  view  learning  cannot  be  ignored.  A large  challenge 
of  program  review  is  to  provide  advanced  courses  that  include  appropriate  and  effective  use  of  current  technologies, 
and  to  create  a dynamic  program  revision  process  that  can  adjust  course  offerings  as  new  and  more  effective 
technologies  and  approaches  become  available. 

Course  Development  and  Outreach  Outcomes 

Transformational  processes  take  time  and  an  intra-departmental  approach  may  create  questions  and  concerns  on  the 
part  of  others  in  the  College.  As  a result,  the  program  revision  committee  has  taken  care  to  present  udpates  at 
departmental  meetings,  to  initiate  meetings  with  administration  and  other  stakeholders,  and  has  responded  to  survey 
data  beyond  the  University.  Curricular  changes  involving  collaborations  have  resulted  in  the  development  of 
academic  coursework  and  flexible  certificates  that  are  planned  for  implementation  Fall  semester  2010. 

Academic  Coursework 

Graduates  of  the  Educational  Technology  M.Ed.  program  should  be  able  to  do  the  following:  ( 1)  demonstrate 
knowledge  of  characteristics  and  issues  surrounding  the  integration  of  technology  for  learning;  (2)  apply  the 
principles  of  learning  to  design,  develop  and  implement  effective  instructional  activities  utilizing  emerging 
technologies;  (3)  apply  the  instructional  design  process  to  design  effective  media  elements  and  instructional 
sequences  to  meet  the  needs  of  diverse  learners;  (4)  critically  analyze  current  trends  and  research  in  educational 
technology;  (5)  demonstrate  competence  in  oral  and  written  professional  communication;  and  (6)  demonstrate 
integrity  and  ethical  professional  behavior  when  designing,  developing  and  implementing  educational  technologies. 

To  complete  a Master  of  Education  degree  with  a concentration  in  Educational  Technology,  one  must  complete  a 
minimum  of  36  hours  of  course  work  to  include  a professional  core.  Educational  Technology  core,  and  specialty 
area  of  concentration.  The  professional  core  consists  of  twelve  credit  hours  including  the  JMU  College  of  Education 
requirements  for  M.  Ed.  programs.  Courses  are:  Inquiry  in  Education,  Curriculum  Theory  and  Instructional 
Models,  Learning  Theories,  and  Principles  of  Instructional  Design.  First  year  students  are  exposed  to  workshops 
that  help  orient  and  prepare  for  the  portfolio  and  qualifying  exam  key  assessments.  The  Educational  Technology 
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program  core  courses  include  Foundations  of  Educational  Technology,  Educational  Technology  Management, 
Design  and  Development  of  Digital  Media,  and  Developing  and  Critiquing  Visual  Literacy.  At  the  conclusion  of 
core  course  work,  students  must  successfully  complete  a qualifying  examination  and  defend  a learning  and  design 
portfolio.  The  Advanced  Educational  Technology  program  courses  are:  User  Interface  Design,  Professional 
Development  in  Educational  Technology,  Advanced  Video  and  Audio  Production,  Data  Visualization,  Games, 
Simulations  and  Virtual  Worlds  for  Learning,  and  eLearning  Applications.  Students  have  a portfolio  defense  at  the 
conclusion  of  the  degree  and  an  inquiry  project  involving  applied  research. 

Flexible  Certificates  & Course  Delivery  Options 

Another  outcome  of  the  curricular  revisions  is  the  design  of  certificate  programs  intended  to  support  individuals  who 
have  an  interest  and/or  need  to  further  their  expertise  in  Educational  Technology,  but  choose  not  to  pursue  the  36 
credit  hour  Masters  degree.  Certificate  courses  are  offered  online  and  completed  hours  may  be  applied  toward  the 
Masters  degree.  Three  certificates  are  currently  available.  They  are:  Educational  Technology,  eLearning,  and 
Educational  Technology  Leadership.  Course  delivery  options  are  to  be  determined  although  online  certificate 
programs  can  inform  and  change  traditional  practices  in  a graduate  program  while  broadening  and  diversifying  the 
student  population.  Distance  learning  is  a relatively  new  consideration  for  both  the  university  and  the  College  of 
Education  during  the  academic  year.  Programs  that  are  moved  substantially  off  campus  or  into  alternative  modes  of 
delivery  (online)  must  be  re-submitted  by  the  university  to  the  accrediting  body  that  is  the  Southern  Association  of 
Colleges  and  Schools,  or  SACS.  It  is  the  desire  of  the  Educational  Technology  committee  to  provide  an  'open' 
approach  to  learning,  whereby  the  learner  has  equal  options  to  meet  face  to  face,  online,  or  a combination  of  both. 
Given  recent  advances  in  real-time  desktop  sharing  and  conferencing,  and  the  ability  to  use  these  tools  in 
conjunction  with  more  high-end  applications  of  classroom  video  conferencing,  the  program  hopes  to  take  advantage 
of  being  able  to  provide  students  with  opportunities  for  course  delivery  choice. 

Conclusion 

The  JMU  Educational  Technology  program  is  moving  through  a transition  prompted  by  interdisciplinary  similarities 
and  the  need  to  serve  a more  inclusive  audience,  including,  but  not  limited  to  K12.  Changes  in  organizational 
structure  and  culture  of  the  College  of  Education  created  a ripe  environment  for  collaborative  efforts  during  this  time 
of  program  review  and  revision.  AECT  provides  the  necessary  framework  of  standards  for  the  program.  This, 
coupled  with  full-time  expertise,  is  allowing  the  program  to  grow  and  to  provide  rich  educational  technology  courses 
and  programs,  while  also  leveraging  itself  as  a foundational  resource  to  the  College  of  Education  at  large. 

Ultimately,  decisions  of  curricular  change  are  dependent  upon  national  standards  and  institutional  classification  and 
expectation.  Guiding  standards  for  educational  technology  are  those  of  AECT  and  ISTE.  The  International  Society 
of  Technology  in  Education  (ISTE)  has  worked  to  establish  technology  proficiency  standards  for  students,  teachers, 
and  administrators  in  PK-12  education.  ISTE’s  National  Educational  Technology  Standards  (NETS)  have  “served  as 
a roadmap  since  1998  for  improved  teaching  and  learning  by  educators.”  ISTE  standards  are  designed  to  help 
measure  proficiency  and  set  benchmarks  for  the  knowledge,  skills,  and  attitudes  needed  to  succeed  in  the  21st 
century.  The  HRD  program  follows  standards  of  the  American  Society  for  Training  and  Development  (ASTD). 

There  are  many  overlaps  in  skills  and  knowledge  among  these  program  standards  and  the  continued  systemic  review 
of  such  standards  and  program  course  offerings  in  both  educational  technology  and  human  resource  development 
will  strengthen  both  programs  and  provide  the  best  of  both  worlds  for  students. 
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Appendix  A 


1.  Your  current  Ed  Tech/IT-related  position  title 

2.  Number  of  years  experience  working  in  this  position 

3.  Describe  typical  practices  in  your  Ed  Tech/IT  workplace  - what  kinds  of  things  are  done  on  a regular  basis? 

4.  In  your  opinion,  what  kind  of  academic/technical/other  skills  are  necessary  to  be  successful  in  your  position? 

5.  List  your  college  degrees,  certificates,  licensure,  etc. 

6.  If  you  obtained  a Bachelor's/Master's/Certificate  in  an  area  related  to  Educational  Technology  or  Instructional 
Technology,  in  what  ways  did  your  program  prepare  you,  and  what  was  missing  from  the  preparation  you  received? 

7.  If  you  were  to  return  to  graduate  school  for  a Master's  degree,  or  refer  someone  to  a Master's  program  in  the 
discipline,  what  kind  of  program  would  be  desirable? 

8.  What  would  be  a desired  course  delivery  method  (f2f,  online,  hybrid,  synchronous,  asynchronous,  etc.)  and  mode 
of  interaction  with  others  in  the  program  (cohort,  non-cohort,  etc.).  Please  explain. 

9.  What  software  and  hardware  do  you  use  on  a regular  basis  to  accomplish  your  professional  goals?  (commercial, 
web-based,  open-source,  etc.) 
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Appendix  B 


Summary  of  informal  survey  results 
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Abstract 

Nieto  (2009)  clearly  frames  a question  that  both  teacher  educators  and  teachers  themselves  ask:  “How  do  teachers 
move  from  simply  surviving  to  actively  thriving  in  the  profession?”  (Educational  Leadership,  February  2009,  p.8). 
Nearly  7,000  students  drop  out  of  school  every  day  (Editorial  Projects  in  Education,  2007).  However,  research 
clearly  tells  us  what  the  qualities  of  an  effective  teacher  are  and  which  instructional  strategies  increase  student 
achievement.  In  this  case  study,  two  research-based  elements  were  combined  to  determine  how  implementing  them 
would  impact  the  pedagogy  of  teachers.  The  results  indicated  when  teachers  were  provided  with  a self-assessment 
metric  of  effective  teacher  characteristics  and  learned  new  research-based  instructional  strategies  in  a collaborative, 
supportive  setting,  their  classroom  instruction  changed  from  teacher-centered  delivery  of  information  to  student- 
centered  learning. 

Keywords:  research-based  instructional  strategies,  effective  teacher  characteristics,  student-centered  learning 
Introduction 

Teachers  learn  just  as  their  students  do:  by  studying,  doing,  and  reflecting;  by  collaborating  with  other 
teachers;  by  looking  closely  at  students  and  their  work;  and  by  sharing  what  they  see.  This  kind  of  learning  cannot 
occur  solely  in  college  classrooms  divorced  from  engagement  in  practice  or  solely  in  school  classrooms  divorced 
from  knowledge  about  how  to  interpret  practice.  Good  settings  for  teacher  learning,  in  both  colleges  of  education 
and  schools,  provide  lots  of  opportunities  for  research  and  inquiry,  for  trying  and  testing,  for  talking  about  and 
evaluating  the  results  of  learning  and  teaching.  (Darling-Hammond,  1997) 

Darling-Hammond  (1997)  pointedly  summarizes  what  classroom  teachers  need  in  order  to  be  highly 
effective  partners  in  the  teaching  and  learning  process.  Teachers  know  this.  Administrators  know  this.  Professors 
know  this.  Yet,  learning  opportunities  that  truly  integrate  the  elements  of  research,  inquiry,  trying,  testing,  and 
collaboration  are  sparse.  The  impact  of  this  dearth  of  meaningful  learning  for  classroom  teachers  is  felt  throughout 
our  school  systems. 

The  meaningful  application  of  skills,  knowledge,  and  dispositions  is  an  essential  process  in  order  for 
teachers  to  internalize  concepts  and  make  them  an  integral  part  of  their  pedagogy.  If  the  classroom  teacher  is 
thoughtful  and  productive  with  her/his  own  content  and  models  it  in  the  delivery  of  instruction,  it  follows  that  the 
students  see  meaningful  teaching  and  learning  in  action. 

Teachers  report  their  classrooms  are  filled  with  apathetic,  bored,  and  passive  students.  At  the  same  time, 
teachers  are  pressured  into  superficially  addressing  content  in  order  to  cover  the  required  state  standards  and  prepare 
students  for  the  end-of-year  tests.  “I  don't  have  time  to  really  teach  or  reach  my  students”  is  lamented  throughout 
teacher  lounges  across  the  nation. 

Overview 

Coburn’s  research  on  teacher  learning  tells  us  that  teachers  respond  to  practices  through  a continuum  of 
action  ranging  from  total  rejection  to  accommodation  (Coburn,  2001).  Teachers  need  to  experience  success  and  to 
see  changes  in  their  own  students’  learning.  One  of  the  keys  in  connecting  to  this  kind  of  thinking  is  to  create 
relevancy  and  results. 

Utilizing  current  research-based  content  is  essential  in  overcoming  questioning  attitudes  toward  learning. 
This  article  integrates  the  work  of  two  powerful  books:  Qualities  of  Effective  Teachers  (2002)  by  James  H.  Stronge 
and  Classroom  Instruction  That  Works  (2001)  by  Robert  J.  Marzano,  Debra  J.  Pickering,  and  Jane  E.  Pollock.  The 
concepts  of  teacher  effectiveness  and  research-based  instructional  strategies  combine  to  form  a foundation  that 
results  in  changes  in  teaching. 
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Study  Participants 

Two  cohorts  of  teachers  in  a master’s  degree  program  studied  the  qualities  of  effective  teachers  and 
instructional  strategies  that  work  over  a one -month  intensive  course.  They  met  twice  a week  for  4.5  hours.  During 
this  time  they  participated  and  interacted  in  a number  of  learning  activities.  They  were  presented  with  the  key 
concepts  through  the  modeling  of  strategies.  They  reflected  on  their  own  teaching  characteristics,  developed  a 
personal  effective  teacher  plan,  studied  and  discussed  research  findings,  debated  the  differences  between  teacher- 
directed  and  student-directed  learning,  developed  new  lessons  and  units,  implemented  new  strategies,  documented 
their  lessons,  shared  what  worked  and  what  didn't,  and  created  portfolio  binders  filled  with  energizing  learning 
activities  that  transformed  their  personal  classroom  instruction  and  learning. 

Multiple  elements  combined  to  produce  transformational  learning.  First  and  foremost  was  the  trust  they 
had  in  each  other  and  their  deeply  felt  desire  to  learn.  The  modeling  of  the  effective  research-based  instructional 
strategies  during  class  time,  collaboration  time  to  discuss  and  share  ideas  and  implementation  techniques,  the 
application  and  practice  of  new  skills,  and  self-reflections  to  improve  practice  also  contributed  to  the  process  of  not 
only  gaining  new  knowledge  and  skills,  but  internalizing  them. 

Method 

At  the  first  session  of  the  course,  each  teacher  completed  the  Teacher  Skills  Self-Assessment  Checklist 
adapted  from  the  Stronge  (2002)  research  (See  Appendix).  After  a self-evaluation  process  where  strengths  and  areas 
for  improvement  were  identified,  a Personal  Teacher  Effectiveness  Plan  (PTEP)  was  created.  The  baseline  self- 
assessment  and  PTEP  were  used  to  determine  personal  growth  and  development  each  week. 

During  Session  1 several  instructional  strategies  from  Classroom  Instruction  that  Works  were  modeled. 
Students  applied:  a.)  identifying  similarities  and  differences  while  discussing  their  educational  philosophies  and 
creating  their  effective  teacher  characteristics  lists;  b.)  note  taking  when  jotting  down  important  concepts;  c.) 
cooperative  learning  when  working  in  small  and  whole  group  activities;  and  d.)  setting  objectives  during  the 
development  of  the  Personal  Effective  Teacher  Plans.  This  intentional  use  of  strategies  from  Classroom  Instruction 
that  Works  was  implemented  throughout  the  course.  Students  were  able  to  experience  the  instructional  strategies, 
internalize  them,  reflect  on  what  worked  (or  didn’t),  and  discuss  how  to  adapt  them  to  their  own  teaching.  They 
were  able  to  understand  the  relevancy  of  the  content  they  themselves  were  learning  and  connect  to  it  on  multiple 
processing  levels  due  to  the  student-centered  learning  environment  that  encouraged  sharing,  discussions,  and  the 
questioning  of  assumptions. 

Connecting  Effective  Teacher  Qualities  to  Instructional  Strategies 

The  next  step  was  to  align  the  effective  teacher  characteristics  with  the  research-based  instructional 
strategies.  This  process  provided  the  framework  for  the  study  of  the  strategies  while  consistently  looping  back  to  the 
PTEP.  This  alignment  provided  both  a visual  and  cognitive  reference  point  for  reflection  to  determine  if  goals  were 
met.  The  class  accomplished  the  alignment  through  a cooperative  learning  assignment,  which  modeled  one  of  the 
effective  instructional  strategies.  The  resulting  chart  (Table  I)  allowed  the  teachers  to  pinpoint  which  instructional 
strategies  were  most  likely  to  increase  their  teacher  effectiveness.  For  example,  if  a teacher  identified  the  area  of 
Monitoring  Student  Progress  as  an  area  of  need,  he/she  would  focus  on  the  strategies  presented  in  the  chapters 
dealing  with  Setting  Objectives  and  Providing  Feedback  as  well  as  Homework  and  Practice. 
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Table  I Alignment  of  Characteristics  and  Strategies 


Effective  Teacher  Quality  Category 

Instructional  Strategy 

Organization  for  Instruction 
Time  management 
Planning  for  instruction 
Communicating  expectations  for  student 
achievement 

Setting  Objectives  and  Providing  Feedback 

Implementation  of  Instruction 
Using  appropriate  questioning 
techniques 

Supporting  active  learning 
Differentiating  instruction 

Nonlinguistic  Representation 
Cue,  Questions,  and  Advance  Organizers 
Generating  and  Testing  Hypotheses 
Identifying  Similarities  and  Differences 
Summarizing  and  Note  Taking 

Management  and  Organizational  Skills 
Creation  of  an  effective  learning 
environment 

Cooperative  Learning 
Collectivism  vs.  Individualism 

Monitoring  Student  Progress 
Feedback 
Homework 

Student  learning  outcomes 

Setting  Objectives  and  Providing  Feedback 
Homework  and  Practice 

Teacher  as  a Person 
Nonacademic  interactions 
Professional  attitude 
Reflective  practice 

Reinforcing  Effort  and  Providing  Recognition 

Collecting  Data 

A qualitative  approach  was  used  for  this  case  study.  Reflection  papers  were  turned  in  at  the  end  of  each 
class  session  that  focused  on  specific  instructional  strategies  paired  with  lesson  plans.  A weekly  journal  entry 
revisited  the  PTEP  and  tracked  their  progress  in  becoming  more  effective  teachers.  The  instructor  kept  a master 
journal  that  tracked  changes  in  pedagogy  and  evidence  of  personal  growth  from  the  weekly  reflections. 

During  the  two  sessions  each  week,  one  or  more  of  the  instructional  strategies  were  presented  followed  by 
a lesson  plan  development  activity.  The  lesson  was  implemented  in  the  classroom  the  following  day  and  the  results 
were  analyzed  during  the  following  class  session.  The  teachers  discussed  in  small  and  large  groups  what  worked  and 
what  needed  revised.  They  each  then  wrote  a reflection  paper  and  turned  that  in  along  with  the  initial  lesson  as  well 
as  the  revised  lesson  attached.  At  the  end  of  each  week,  the  Reflective  Journals  were  sent  via  e-mail  to  the  instructor. 
They  included  personal  progress  towards  the  goals  generated  from  the  self-assessment  task. 

Evidence  of  Growth 

The  power  of  collaboration  in  education  has  been  written  about  with  great  authority  (Darling-Hammond. 
1997).  One  of  the  ongoing  frustrations  of  being  a classroom  teacher  is  the  feeling  of  isolation  that  happens  on  a daily 
basis  (Turley,  2005). 

One  of  the  many  positive  aspects  of  being  a member  of  a cohort  is  the  time  for  collaboration,  sharing,  and 
teacher  talk.  The  teachers  who  participated  in  this  study  talked.  They  shared  what  worked  and  what  didn't.  They 
enthusiastically  explained  how  their  teaching  was  changing.  They  asked  for  new  ideas  from  their  colleagues.  They 
designed  and  developed  new  learning  activities.  Their  reflective  papers  and  journals  clearly  showed  evidence  of 
growth  and  change  in  their  pedagogy. 

All  thirty-one  teachers  reported  that  their  teaching  had  improved  by  implementing  the  research-based 
instructional  strategies.  They  reported  feeling  more  confident  in  how  they  defined  themselves  as  effective  teachers 
and  stated  when  implementing  the  new  skills  and  knowledge  their  own  students  were  more  engaged  in  learning. 
Conclusion 

Sonia  Nieta  (2009)  poses  the  question:  How  do  teachers  move  from  simply  surviving  to  actively  thriving  in 
the  profession?  From  the  collaborative  work  done  by  the  teachers  in  this  study,  the  answer  would  appear  to  be,  by 
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understanding  the  connection  between  effective  teacher  characteristics  and  research-based  effective  instructional 
strategies.  The  teachers  first  clearly  identified  their  areas  for  improvement  and  then  learned  and  applied  specific 
strategies  to  improve  their  instruction  together.  The  results  included  changes  in  attitudes  and  pedagogy.  The  teachers 
talked  about  a return  to  a love  of  and  enthusiasm  for  teaching. 

Suggestions  for  Future  Research 

Action  research  is  a fertile  avenue  to  provide  meaningful  information  about  what  is  taking  place  in  the  field 
of  teaching  today.  This  case  study  was  conducted  with  two  cohorts  of  teachers  focusing  on  how  the  combination  of 
two  powerful  concepts  could  merge  to  support  a change  in  pedagogy.  Each  month,  quarter,  or  semester  we  have  the 
opportunity  to  study  what  strategies  improve  teaching  instruction  and  in  turn,  what  creates  engaging  learning 
environments.  More  research  in  this  area  will  help  us  to  become  more  effective  teachers. 
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Appendix 


Teacher  Skills  Self-Assessment  Checklist 

The  following  checklist  is  adapted  from  Qualities  of  Effective  Teachers,  James  H.  Stronge. 

Directions: 

1.  Use  the  following  scale  to  score  your  qualities  as  a teacher. 

2.  Then,  review  the  checklist  and  highlight  the  HE  and  AE  qualities  in  blue. 

3.  Go  on  to  highlight  the  E qualities  in  green. 

4.  And  finally,  highlight  the  ME  and  IE  qualities  in  yellow. 

5.  Reflect  on  the  ME  and  IE  qualities.  What  skills  and  knowledge  do  you  need  to  acquire  to  move  these  into  the  E 
rating  or  above?  Identify  those  qualities  you  want  to  improve. 

6.  Create  an  Effective  Teacher  Improvement  Plan.  Indicate  specific  actions  you  will  take  to  move  the  identified 
qualities  into  a higher  rating.  Place  this  plan  in  your  portfolio.  The  expectation  is  that  you  will  identify  an  action 
each  week  and  work  on  improving  your  teacher  effectiveness. 

HE  = Highly  Effective  An  HE  rating  indicates  that  the  teacher  is  an  expert  and  has  mastered  this  quality. 

AE  = Approaching  Effective  An  AE  rating  indicates  a teacher  who  exhibits  this  quality  90%  of  the  time. 

E = Effective  An  E rating  indicates  a teacher  who  demonstrates  this  quality  at  least  75%  of  the  time  and  gets 
effective  results. 

ME  = Marginally  Effective  A ME  rating  indicates  a teacher  who  infrequently  (70%  or  less  of  the  time) 
demonstrates  the  quality  and  needs  to  gain  additional  skills  in  this  area. 

IE  = Ineffective  An  IE  rating  means  the  teacher  does  not  possess  the  quality. 

N/A  = The  quality  is  not  applicable  to  the  subject  matter. 


Quality 

Indicators 

HE 

AE 

E 

ME 

IE 

N/A 

Teacher  as  a Person 

Listens  actively 

Develops  a personal  relationship/interest  in  student 
while  maintaining  professional  boundaries 

Values  student  input 

Exhibits  love  of  teaching 

Shows  passion  for  content  area 

Enjoys  teaching  and  learning 

Seeks  new  knowledge  and  skills 

Engages  in  reflective  practice  to  improve  teaching 

Sets  high  expectations  for  personal  classroom 
performance 

Organizing  for 
Instruction 

Carefully  prepares  meaningful  lessons  aligned  to 
standards 

Selects  a variety  of  teacher-directed  and  student- 
directed  strategies  and  techniques 

Prepares  all  materials  in  advance 

Organizes  sequencing  of  lesson  for  effective 
presentation 

Sets  clear  learning  outcomes  aligned  to  standards  and 
explains  the  linkage  to  students 
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Sets  a pace  that  maintains  momentum  throughout  the 
class  period 

Implementation  of 
Instruction 

Optimizes  use  of  instructional  time 

Varies  research-based  instructional  strategies, 
learning  activities,  and  assignments  to  engage  and 
motivate  learners 

Paces  learning  and  is  able  to  adapt  as  the  situation 
demands 

Uses  appropriate  questioning  techniques  to  scaffold 
learning  and  increase  critical  thinking 

Connects  prior  knowledge  to  new  concepts 

Frequently  checks  for  understanding  using  a variety 
of  techniques 

Sets  high  expectations  for  learning  which  includes 
sttessing  student  responsibility  and  accountability 

Gives  clear  examples 

Differentiates  instruction 

Changes  plan  as  needed  based  on  successes  and 
challenges 

Celebrates  student  learning  in  a positive,  reinforcing 
manner 

Management  and 
Organizational  Skills 

Prepares  plans  and  materials  in  advance 

Involves  students  in  creating  classroom  expectations 

Is  fair  and  consistent  in  using  discipline 

Establishes  a classroom  routine  that  creates  a safe, 
yet  welcoming  learning  environment 

Balances  variety  and  challenge  in  selection  of 
learning  activities 

Uses  space  to  maximize  learning  and  to  encourage 
interaction 

Maintains  a highly  effective  learning  environment 
that  values  student  learning 
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Monitoring  Student 
Progress 

Clearly  explains  the  purpose  for  assignments  and 
homework 

Provides  timely  feedback  based  on  clear  grading 
rubrics  to  improve  student  performance 

Provides  opportunities  for  re-teaching  concepts  as 
needed 

Knows  the  strengths  and  needs  of  each  student  based 
on  ongoing  continuous  assessment 
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A Review  of  Factors  Impacting  Online  Healthcare  Education: 
Guidelines  for  Enhancement 

Jun  Fang,  Scott  Schaffer 
Purdue  University 

This  literature  review  investigates  and  classifies  key  content  delivery  methodologies  and  user 
characteristics  that  are  believed  to  have  high  impact  on  the  quality  of  online  healthcare  education.  The  purpose  of 
this  review  is  to  identify  significant  user  characteristics  and  methodologies  such  as  information  delivery 
technologies,  health  assessment  tools  and  web-page  interface  element  accommodations  in  the  health  information 
delivery.  This  information  will  inform  the  instructional  and  graphic  design  of  learning  systems  for  this  population. 

Online  healthcare  education  has  been  a primary  source  of  disseminating  and  receiving  health  information 
for  both  consumers  and  professionals  for  the  past  several  years  (Liu,  2007;  Morrell,  Mayhorn  & Bennett,  2002). 
While  the  increase  of  online  healthcare  sites  make  it  worthy  of  investigation  and  the  potential  of  utilizing  new  web 
technologies  to  facilitate  healthcare  is  just  emerging  ( Morgan,  2002),  much  of  the  current  information  found  online 
is  informal  and  insufficient  in  scope.  For  this  review,  the  web  sites  of  ten  of  the  "best"  hospitals  according  to  the 
honor  roll  in  U.S.  News  2009  were  analyzed  to  better  understand  the  type  and  scope  of  content  being  delivered  to  the 
public.  The  results  show  that  many  of  these  web  sites  are  crowded  with  information  but  do  not  follow  basic  design 
guidelines  for  facilitating  efficient  health  care  information  delivery  (see  appendix).  Mead,  Lamson  & Roger  (2002) 
extensively  reviewed  current  health-oriented  websites  and  pointed  out:  “many  exiting  web  sites  have  characteristics 
that  would  make  them  virtually  unusable  by  the  average  older  adults  and  by  many  young  adults  as  well”  ( Meadd, 
Lamson  & Roger,  2002,  P.  96).  This  review  focuses  on  the  quality  of  online  health  care  information  dissemination 
by  investigating  and  classifying  factors  helping  with  quality  improvement. 

The  quality  of  online  health  care  education  is  profoundly  impacted  by  content  delivery  methodologies  such 
as  information  delivery  technologies  (Hollandsworth,  2007;  Boulos,  Maramba  & Wheeler,  2006;  Heinich,  et  al., 
1993;  Ma,  Lee,  Du  & McCahill,  1998;  Osheroff,  1994;  Wei  & Levkoff,  2000;  Siau.K,  2003;  Mehrabi,  et  ah,  2000), 
health  assessment  tools  (Kitchie,  2003;  Shapiro,  et  ah  1996;  Siau,  2003;  Bensley,  R.,  & Lewis,  J,  2002  ),  health 
information  accommodations  (Shepperd,  Charnock,  & Gann,  1999;  Risk,  & Dzenowagis,  2001,  Eysenbach,  et  al, 
2002),  and  user  characteristics  including  the  characteristics  of  patients,  primary  care  providers/family  members,  and 
other  healthcare  professionals  (Ewler  & Sinnette,  1986;  Mead,  Lamson  & Rogers,  2002;  Wetle,  2002;  Siau,  2003; 
Bastable,  2003;  Dryfoos,  1990;  Diclemente,  Hansen,  & Ponton,  1996;  James,  & Gabe,  1996;  Kitchie,  2003; 
McCormick,  & Gilson-Parkevich,  1979;  Treweek,  1996;  Cline  & Haynes,  2001  ).  This  review  focuses  on  each  of 
these  issues  and  is  divided  into  two  categories:  1)  content  delivery  methodologies;  and  2)  user  characteristics. 

Content  delivery  methodologies 


Information  Delivery  Technologies 
1.  Podcast 

Since  podcasts  first  appeared  in  2004  (Hollandsworth,  2007)  they  have  been  widely  adopted  in 
entertainment  and  education  sectors.  A podcast  is  a digital  media  file  (audio  or  video-vodcasts)  distributed  via  the 
Internet  for  the  audience  to  utilize  whenever,  wherever  and  however  they  want  ( Boulos,  Maramba  & Wheeler, 
2006).  Podcasts  provide  users  with  a flexible  environment  to  listen  to  content  through  a number  of  delivery  systems 
including  their  phone,  computer,  or  portable  MP3/MP4  players.  Such  audio  media  is  desirable  for  many  people 
because  “the  impact  of  the  spoken  word  is  frequently  greater  than  that  of  print”.  (Heinich,  et  al.,  1993,  pl63).  Audio 
materials  are  cheap  to  produce  and  distribute,  and  easy  to  access.  Some  universities  use  podcasts  within  online 
education  to  deliver  lectures  and  review  assignments.  Results  indicated  that  podcasts  minimized  technological 
support  demands  and  empowered  learners  to  self  manage.  Learners  found  them  easy  to  use  and  benefited  from  the 
flexibility  in  assimilating  large  amounts  of  information  (Hollandsworth,  2007).  Computer-based  interactive 
programs  using  audio-based  instructions  have  been  found  to  be  at  least  as  effective  as  other  forms  of  content 
delivery) Heinich,  et  al.,  1993).  Finally,  podcasts  are  good  alternatives  for  users  suffering  visual  impairment  ( 
Heinich  et  al.,  1993).  A major  challenge  with  podcasts  are  that  audio  and  video  files  take  a long  time  to  download 
and  require  sufficient  bandwidth  ( Boulos,  Maramba  & Wheeler,  2006). 
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2.  Video  clips 


The  application  of  video  clips  online  has  been  very  popular  for  global  information  distribution  (Ma,  Lee, 

Du  & McCahill,  1998).  The  special  attributes  of  video  streams  -manipulation  of  space  and  time  enable  learners  to 
view  phenomena  in  microcosm  and  macrocosm  and  experience  the  process  of  an  event  by  compressing  or 
expanding  the  time  that  it  takes  to  occur.  Researchers  outlined  the  advantages  of  the  application  of  video  clips  based 
on  their  attributes:  a)  motion  of  images;  b)  process  of  operation;  c)  safe  observation;  d)  skill  learning;  e) 
dramatization;  f)  affective  learning;  g)  problem  solving;  h)  cultural  understanding;  i)  establishing  commonality. 
(Heinich,  et  ah,  1993).  Video  clips  are  not  only  an  attractive  feature  for  web  pages;  their  advantages  make  them  a 
powerful  tool  to  effectively  disseminate  information  and  education  online. 

3.  Online  Discussion  Groups 

Online  discussion  groups  such  as  blogs  and  forums  are  attracting  a large  and  dedicated  readership  as  they 
encourage  people  in  knowledge  sharing,  reflection,  and  debate  ( Boulos,  Maramba  & Wheeler,  2006).  Osheroff 
(1994)  stated  that  “online  forums  are  an  increasingly  important  source  for  both  physicians  and  patients  seeking 
education,  guidance  in  clinical  decision  making  and  interpersonal  connections”  (Osheroff,  1994,  para.  2).  It  is  easier 
for  either  patients  or  care  givers  to  find  coping  strategies  and  emotional  support  in  a group  with  others  who  share  the 
same  situation  (Wei  & Levkoff,  2000,  p314;  Osheroff,  1994,  para.  4;  Siau,K,  2003).  Online  discussion  forums 
provide  such  stages  for  health  information  users  to  “discuss  daily  challenges  and  stresses  and  exchange  ideas  for 
dealing  with  a wide  variety  of  problems”  (Levkoff,  2000,  p3 14).  Other  main  methods  to  conduct  online  discussions 
are  blogs,  list  servers  and  newsgroups.  Less-expensive  Internet  access,  more  useful  information,  more  practical 
advice  and  more  accessible  emotional  support  make  online  discussions  more  and  more  popular.  In  another  study, 
Culver  & colleagues  (1997)  analyzed  messages  displayed  on  Internet  bulletin  board  and  found  out  that  one 
challenge  facing  users  was  the  accuracy  of  the  health  information  (Culver,  1997).  Since  people  from  around  the 
globe  can  share  their  advice  and  strategies  in  the  online  discussion  platforms,  the  information  calls  for  strict  scrutiny. 

4.  Animations,  graphics  and  slide  shows 

We  are  a visual  society.  Studies  indicate  that  the  interactive  use  of  multi-media  components  such  as 
graphics,  animation,  sound,  and  digital  slide  shows  can  theoretically  facilitate  the  learning  process(  Mehrabi,  et  al., 
2000).  Moreover,  moving  images  “have  an  obvious  advantage  over  other  visual  media  in  portraying  concepts’^ 
p.220),  such  as  a first  aid  demonstration.  When  used  appropriately,  animations  and  graphics  will  facilitate  learning 
(Ackerman  et  al.,  2006).  Heinich  et  al.  also  pointed  out  that  graphics  and  animations  were  not  only  helpful  for  young 
children  but  also  for  adult  learners  (Heinich  et  al.,  1993). 

Health  Assessment  Tools 

Kitchie  (2003)  pointed  out  that  assessment  was  essential  to  determine  learning  needs  in  any  of  the 
cognitive,  affective  or  psychomotor  domain  and  assessment  of  people’s  health  status  was  vital  to  determine  the 
amount  of  energy  available  as  well  as  present  comfort  level,  both  of  which  profoundly  impact  the  cognitive 
procedure  of  information  assimilation  ( Kitchie,  2003).  Health  Assessments  can  help  identify  the  signs  of  symptoms 
for  many  physical  and  mental  health  conditions,  as  well  as  social  and  behavior  issues  (Shapiro,  et  al.  1996). 
Currently  most  commonly  used  health  assessment  tools  online  are  Health  Calculators,  Self-Assessment  and  Quizzes. 
Patient  oriented  calculators  and  assessment  tools  help  answer  common  questions  like  "What  are  the  symptoms?”. 
They  cover  common  symptoms  of  health  conditions,  addictions,  and  other  behaviors.  Studies  believe  that  the  more 
patients  know,  the  more  they  can  participate  with  their  physician  in  planning  their  own  health  goals  (Kitchie,  2003; 
Siau,  2003).  Interactive  quizzes  and  other  tools  will  help  patients  pinpoint  problem  areas,  assess  their  risks  for 
certain  diseases  and  conditions,  and  assist  in  improving  health  and  well-being.  Health  assessment  tools  are  not  only 
helpful  for  patients;  they  assist  practitioners  in  developing  interventions  to  promote  health  (Bensley,  R.,  & Lewis,  J, 
2002) 
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Health  Information  and  Web-page  Interface  Accommodations 

Issues  concerning  online  health  information  have  emerged  (Shepperd,  Charnock,  & Gann,  1999;  Risk,  & 
Dzenowagis,  2001,  Eysenbach,  et  al,  2002)  with  the  proliferation  of  the  Internet  and  the  massive  growth  of  online 
health  information.  The  convenient  access  to  myriad  of  health  information  is  companied  with  potential  dangers.  Risk 
and  Dzenowagis  (2001)  pointed  out  that  numerous  surveys  and  studies  had  painted  a picture  of  dubious  information 
quality,  widespread  practice  of  fraud  and  the  risk  of  exposure  of  the  public  to  harm  (Risk,  & Dzenowagis,  2001). 
Information  with  regard  to  a)  symptoms  and  conditions,  b)  remedies  and  treatment,  and  c)  professional  reports  and 
recommended  books  will  be  good  resource  for  users  to  take  advantage  of. 

A lot  of  time  is  spent  modifying  web  page  interface  designs  to  facilitate  the  effective  use  of  online 
information  among  a variety  of  users  (Echt,  2002;  Mead  et  al.,  2002;  Ackerman,  2006).  User-centered  web  page 
design  is  considered  to  be  the  best  approach.  “It  is  not  enough  to  design  for  our  own  effective  use  and  aesthetic 
satisfaction”  rather  effective  health  information  web  sites  should  be  designed  to  “maximize  the  likelihood  that  it  can 
be  effectively  sensed,  perceived,  understood,  and  consequently  learned  and  applied”  (Echt,  2002,  pp.201). 

1.  Visual  adjustment 

Research  shows  that  it  may  be  better  for  older  users  and  users  with  visual  impairment  to  use  at  least  14- 
point  fonts,  and  user  should  be  allowed  to  individualize  text  size  (Echt,  2002,  Mead  et  al.,  2002).  Also,  familiar  fonts 
such  as  sans  serif  font  types  that  are  at  least  12-points  should  be  used  (Ackerman,  2006).  High  contrast  text  is 
recommended  in  web  designs.  Mead  (2002)  suggested  that  dark  text  on  a light  background  was  ideal  to  improve 
reading  performance  (p98).  Studies  showed  that  patterned  backgrounds  would  slow  users  reading  speed  by  32 
percent  (Ackerman,  2006).  Since  “most  learners  prefer  colored  visuals”  (Heinich  et  al.,  1993,  p 68),  color  used  on 
web  pages  is  an  attention-attracting  feature.  However,  they  should  be  used  appropriately,  or  they  will  potentially 
annoy  users.  Web  design  guidelines  suggests  that  color-coding  should  be  used  when  the  coding  scheme  can  be 
easily  understood,  that  bold  for  headers  and  key  words  to  emphasize  importance  and  that  the  same  way  or  color 
should  be  used  to  highlight  the  same  information  ( Ackerman,  2006;  Echt,  2002). 

Heinich  et  al.  (1993)  stated  that  young  users  prefer  simple  visuals,  whereas  older  prefer  more  complex 
visuals.  Appropriately  used  graphics  and  animations  enhance  information  assimilation.  Web  users  are  helped  by 
instruction  that  are  combined  by  text  and  text-relevant  illustrations  ( Echt,  2002).  However,  they  are  supposed  to  be 
used  in  a meaningful  way,  which  means  “use  graphic  and  animations  only  when  they  help  to  convey,  or  are 
supportive  of,  the  Web  site’s  message  or  other  content”  (Archer  man,  2006).  Meaningless  graphics  and  animations 
or  abusing  graphics  can  only  frustrated  users  and  slow  the  web  page  download  times.  Research  suggested  reducing 
unnecessary  graphics  and  animations  and  providing  high  contrast  and  high  resolution  ones  for  older  users)  Mead  et 
al.,  2002).  . 

2.  Audio/video  streaming 

Multimedia  such  as  audio  and  video  can  easily  capture  users’  attention  so  it  is  effective  methods  to 
disseminate  important  information.  However,  they  are  distracting  elements  if  there  are  no  clear  and  useful  reasons  of 
using  them  (Archerman,  2006).  A myriad  of  Web  users  including  older  people  are  not  technology  guys.  Mead 
(2002)  revealed  that  some  adjustment  on  audio/video  incorporation  in  Web  pages  would  have  salient  effect  on  the 
web  users.  The  modifications  concerning  audio/video  streaming  are  listed  as  follows: 

• Avoid  unnecessary  use  of  audio/video  streaming; 

• Avoid  complex  streaming  software; 

• Use  the  same  kind  of  software  for  all  the  audio/video  streaming; 

• Do  not  require  users  to  download  or  install  plug-ins; 

• Limit  the  download  time  and  improve  the  video/audio  quality. 

3.  Disability  accommodations. 

In  another  study,  Gonzalez  etc  (2003)  revealed  that  people  with  disabilities,  especially  people  with  visual 
disabilities,  found  it  difficult  to  retrieve  information  from  the  net  due  to  the  fact  that  most  web  pages  placed  their 
emphasis  on  aesthetics  and  attractiveness  rather  than  functionality  and  simplicity.  During  the  study,  Gonzalez  and 
colleagues  explored  the  project  KAI  (Kit  for  the  Accessibility  to  the  Internet),  introducing  an  accessibility 
measurement  module  to  assess  the  accessibilities  of  existing  web  pages  and  provide  guidelines  for  web  designers. 
The  study  indicated  that  web  designers  should  take  into  account  the  accessibility  of  the  web  content  and  the 
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utilization  of  some  tools  such  as  WebTouch,  which  would  make  online  information  more  accessible  for  a larger 
range  of  users.  According  to  American  Disabilities  Act,  information  provided  via  Web  must  be  accessible  to  people 
with  a wide  range  of  sensory  and  physical  disabilities.  Echt  (2002)  also  pointed  out  that  such  assistive  technologies 
as  screen  magnification  programs,  screen  readers,  and  refreshable  Braille  displays  made  it  possible  for  most  people 
with  disabilities  to  surf  the  Internet. 


User  Characteristics 

There  are  various  users  of  online  healthcare  information  ranging  from  patients  to  care  givers  and 
professional.  Thus  the  point  for  effective  information  design  is  to  have  users  clearly  in  mind.  Ewler  & Sinnette 
(1986)  stated:  “To  plan  effective  health  education,  the  first  step  is  to  identify  the  user  and  their  characteristics” 
(Ewler  & Sinnette,  1986).  A lot  of  other  authors  also  mentioned  the  important  role  that  intended  user  characteristics 
play  in  the  success  of  healthcare  education  (Mead,  Lamson  & Rogers,  2002;).  Tailoring  information  and  web 
interface  to  fit  with  users’  age,  existing  knowledge  and  experiences  is  essential.  Based  on  the  reports  from  APHA 
(American  Public  Health  Association,  2009)  and  American  Heart  Association  (2008),  diabetes  and  heart  diseases  are 
top  killers  threaten  American’s  life.  Thus,  the  user  characteristic  description  focuses  on  these  two  patient 
populations:  patients  with  heart  disease  and  those  with  juvenile  diabetes. 


Patient 

Wetle  (2002)  revealed  in  his  study  that  both  older  and  younger  persons  had  powerful  motivators  to  seek 
information  of  maintaining  good  health  and  preventing  disease  (Wetle,  2002,  p.  5).  Issues  regarding  diseases  are  the 
big  issues  in  most  people’s  life.  The  Internet  allows  patients  to  gain  up-to-date  information  and  benefit  from  online 
support  groups  sharing  the  same  diseases  (Siau,  2003). This  review  examines  characteristics  for  two  typical  patient 
populations:  heart  disease  and  juvenile  diabetes. 

1.  Age 

Juvenile  Diabetes  also  is  known  as  type  I diabetes  or  insulin-dependent  diabetes  mellitus  (IDDM). 
Although  juvenile  diabetes  can  occur  at  any  age  (Medical  Encyclopedia,  2008),  it  is  more  common  for  patient  at  the 
age  19  and  younger.  Patients  with  Juvenile  Diabetes  have  to  walk  through  their  daily  life  including  obtaining  health 
care  information  with  the  physical  and  emotional  changes. 

While  IDDM  mostly  attacks  juveniles  aged  under  19,  heart  diseases  of  various  kinds  are  the  number  one 
killer  in  American,  especially  for  older  people  over  65  ( American  Heart  Association).  According  to  the  American 
Heart  Association,  almost  150,00  Americans  killed  by  heart  disease  each  year  are  around  the  age  of  65,  and  one  out 
of  every  20  people  below  the  age  of  40  has  heart  disease.  Patients  with  heart  diseases  are  threatened  by  death  while 
facing  the  ongoing  physical  and  psychological  changes  (Bastable,  2003). 

2.  Physical  and  psychological  conditions 

Adolescence  are  known  to  be  among  the  nation's  most  at-risk  populations  (Dryfoos,  1990),  and  an 
understanding  of  their  characteristics  is  crucial  for  patient  education  to  be  effective  ( Bastable,  2003).  Bastable 
(2003)  and  Dantzer,  et  al.(2003)  revealed  the  physical  and  psychological  characteristics  influencing  adolescence 
juvenile  patients: 

• Experiencing  biological  and  psychological  alternations 

• Maturation  of  cognitive  development 

• Psychological  development  resulting  in  conflict,  toleration  and  alternation  between 

adolescents  and  authority  such  as  parents,  doctors. 

• Easier  exposure  to  psychological  disorders,  especially  anxiety  disorders  and  eating 

disorders. 

• Showing  signs  of  depression,  lack  of  will  and  learned  helplessness. 

The  maturation  of  adolescent  patients’  cognitive  development  enables  them  to  understand  the  concepts  of 
health  and  illness,  the  multiple  causes  of  diabetes,  the  impact  of  variables  on  disease  control  and  the  ideas  of  health 
promotion,  while  their  psychological  conditions  such  as  anxiety  and  fear  negatively  interfere  with  their  ability  to 
obtain  information  (James,  & Gabe,  1996). 
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In  addition  to  the  threats  from  heart  diseases,  the  patients,  aged  35  to  60  or  older,  are  undergoing  the 
physical  and  psychological  changes,  which  have  a great  impact  on  the  efficiency  of  obtaining  health  related 
information.  Bastable  (2003)  pointed  out  that  these  patients  were  faced  with  a lot  of  physical  changes,  such  as 
metabolism  slowing  down,  endurance  and  energy  level  lessening,  hearing  and  visual  acuity  diminishing,  which 
greatly  influence  their  learning  capacity.  On  the  other  hand,  the  psychological  alternations  associated  with  aging, 
such  as  developing  concerns  for  their  children,  recognizing  the  psychological  changes  in  themselves,  dealing  with 
the  new  roles  of  being  a grandparent,  affect  their  organ  functioning,  ultimately  reducing  their  energy  level  ( 

Bastable,  2003). 

Thus,  more  online  information  should  focus  on  assisting  the  patients  in  coping  with  stress  and  maintaining 
optimal  health  status  ( Bastable,  2003).  Another  studies  stated  that  patients  tended  to  focus  their  energy  on 
information  related  to  treatment,  test  and  minimizing  pain  or  other  discomfort  and  so  they  did  not  have  energy  for 
learning  of  product  introduction,  life  style  (Kitchie,  2003). 

Family  members  as  health  care  providers 

Care  providers/  support  group  are  a great  part  of  health  care  information  seekers.  Care  providers/support 
group  mainly  consist  of  family  members,  volunteers,  healthcare  workers  and  friends  (Ewler  & Simnett,  1986).  For 
the  heart  failure  group,  the  age  is  probably  like  65-85  so  the  most  likely  care  providers  would  be  their  children  who 
would  be  about  40-55.  Juvenile  diabetes  patients  are  more  apt  to  care  themselves,  but  their  parents  or  aunts/uncles 
may  also  provide  support,  seeking  health  care  information.  Thus  the  age  range  will  be  35-55  for  parents  and  others 
(Bastable,  2003). 

1 . Experience  and  knowledge 

Health  care  experience  and  knowledge  have  a great  impact  on  selecting  and  acquiring  information. 
Generally  speaking,  most  family  members  or  friends  as  health  care  providers  / support  group  have  less  expertise 
compared  to  professionals.  However,  many  family  members  as  health  care  providers  mybe  unaware  of  the 
knowledge  they  lack  or  may  be  reluctant  to  admit  their  deficits  (McCormick,  & Gilson-Parkevich,  1979).  a lot  of 
information  is  needed  to  raise  health  consciousness,  help  to  make  decision,  facilitate  attitude  change  (Ewler  & 
Simnett,  1986) 

2.  Emotional  attitude  and  motivation 

Treweek  ( 1996)  pointed  out  that  understanding  the  world  as  an  emotional  space  had  been  a central  concern 
in  health  care  work  (Treweek,  1996).  Gagne  pointed  out  that  attitudes  were  “mental  states  that  predispose  the 
individual  towards  some  choice  of  personal  action”  (p.  143).  Users’  emotional  attitudes  are  vital  in  selecting  and 
accepting  information. 

Studies  ( Ewler  & Simnett,  1986;  Bastable,  2003)  revealed  the  special  emotional  attitude  care  providers/  support 
group  encountered: 

• . Feelings  of  overburdened,  underappreciated  and  hopelessly  inadequate 

• . Pressures  of  caring  for  children  and  parents  are  overwhelming 

• . Feelings  of  depression,  stress  and  fatigue 

• . Expressing  a form  of  grief,  at  the  thought  of  losing  loved  ones 

Such  emotion  as  anxiety  and  grief  negatively  influenced  their  attitude  to  acquire  information  ( Kitchie, 
2003),  and  thus  the  effectiveness  of  health  care  information  may  be  enhanced  by  introducing  some  information  on 
diminishing  their  anxiety  and  grief,  enhancing. 

However,  some  degree  of  anxiety  is  a motivator  to  learn  (Gagne,  1991).  Their  love  for  their  parents  or  children  is 
strength  for  information  acquiring  and  they  believe  that  “ doing  so  is  a gift  that  they  will  always  cherish”(  Ewler  & 
Simnett,  1986). 

Healthcare  Professionals 

Cline  and  Haynes  (2001)  pointed  out  that  not  only  patients  and  caregivers,  but  also  professionals 
increasingly  engaged  in  interactive  online  health  communications  (Cline  & Haynes,  2001).  The  Internet  is  potential 
platform  for  health  care  professionals  to  obtain,  exchange  and  advertise  their  expertise.  Healthcare  professionals  find 
the  Internet  more  convenient  to  provide  patients  with  good  and  instant  service. 

The  Internet  allows  physicians’  access  to  immediate  information  without  having  to  shift  through  piles  of  files;  it 
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enhances  quick  and  seamless  interaction  between  physicians  and  patients(Siau,  2003);  it  is  more  convenient  tool  for 
prescribing,  scheduling  appointment,  sending  reports,  reminding  and  alerting  patients  about  their  health  status;  it 
makes  professionals  capable  of  responding  to  patients  inquiries  more  conveniently  because  of  the  asynchronous 
nature;  it  is  even  potential  space  for  e-commerce,  extending  professionals  capacity  to  profit  from  selling  various 
commodities  ( Kassirer,  2000). 


Conclusion 

The  conclusion  drawn  from  these  reviewed  studies  implies  that  both  content  delivery  methodologies  like 
information  delivery  technologies,  health  assessment  tools  , health  information  accommodations  and  web  page 
interface  accommodations  (Hollandsworth,  2007;  Boulos,  Maramba  & Wheeler,  2006;  Heinich,  et  ah,  1993;  Ma, 
Lee,  Du  & McCahill,  1998;  Osheroff,  1994;  Wei  & Levkoff,  2000;  Siau,K,  2003;  Culver,  1997;  Mehrabi,  et  ah, 

2000  ) and  user  characteristics  including  the  characteristics  of  Patient,  Care  provider/Family  member  and  Healthcare 
professionals;  Ewler  & Sinnette,  1986;  Mead,  Lamson  & Rogers,  2002;  Wetle,  2002;  Siau,  2003;  Bastable,  2003; 
Dryfoos,  1990;  Diclemente,  Hansen,  & Ponton,  1996;  James,  & Gabe,  1996;  Kitchie,  2003;  McCormick,  & Gilson- 
Parkevich,  1979;  Treweek,  1996;  Cline  & Haynes,  2001  ) have  a great  influence  on  the  quality  of  online  heath  care 
education.  The  researchers’  understanding  of  these  factors  and  the  existing  research  are  crucial  for  web  designers  in 
designing  future  effective  and  qualified  healthcare  related  web  sites.  In  order  to  improve  the  quality  of  online  health 
information  dissemination,  a variety  of  content  delivery  technologies  can  be  manipulated  with  consideration  to  user 
characteristics.  The  users’  awareness  of  these  factors  is  also  significant  in  assisting  themselves  in  health  promotion 
and  disease  prevention. 
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Abstract 

This  paper  introduces  you’re  Skype  - free  videoconferencing  software  to  enhance  collaborative 
learning.  Imagine  your  class  is  working  on  a multicultural  project  with  another  a class  in  Spain. 

You  want  to  set  up  videoconferencing  for  the  students  to  learn  - first-hand  - about  aspects  of 
culture  and  language  from  each  other.  Without  expensive  equipment,  how  can  you  make  this  a 
reality?  We  want  to  introduce  you  to  Skype  - an  easy-to-use  tool  that  will  allow  you  to  accomplish 
this  and  many  other  learning  activities. 


Background  Information 

As  teachers  trying  to  broaden  your  students’  world,  a number  of  teaching  strategies  can  be  embedded  to  the 
curriculum.  For  example,  you  can  invite  a guest  speaker  from  a multicultural  background  to  talk  about  their  culture, 
or  show  a video  on  a topic  relevant  to  the  unit  you  are  teaching.  But  what  if  you  want  to  work  on  a science  project 
with  a partner  class  a hundred  miles  away? 

Creative  teachers  are  always  finding  ways  to  integrate  new  teaching  strategies  to  help  their  students  gain  knowledge, 
but  there  always  seem  to  be  stumbling  blocks.  They  can  be  as  minor  as  a guest  speaker’s  availability  not  aligning 
with  your  class  schedule,  or  as  major  as  not  having  the  budget  to  fund  a file  trip  or  other  activity. 

What  could  be  better  to  explore  Spanish  cultural  practices  than  taking  your  students  to  Spain?  Impossible  you  say? 
We  would  like  to  introduce  you  to  Skype  - a technology  available  today,  that  would  allow  you  to  overcome  these 
obstacles  and  bring  resources  to  your  students  in  real  time  that  you  never  thought  possible. 

What  is  Skype? 

Skype  is  a freeware  communication  tool  that  was  launched  in  Estonia  in  2003,  and  was  purchased  by  eBay  in  2005 
for  $2.6  billion  dollars.  Skype  is  an  effective  no-cost  tool  that  allows  you  to  remotely  talk  with  and  see  others  via  the 
Internet  from  anywhere  in  the  world.  Once  you  download  Skype  to  your  computers  it  allows  you  to  communicate 
with  people  all  over  the  world  for  FREE,  with  an  Internet  connection. 

With  only  a PC,  Internet  connection,  microphone,  speakers  and  Skype  you  can  talk  or  make  video  calls  (with  a 
webcam),  send  instant  messages  and  files,  across  the  street  or  across  the  globe  without  costing  a penny.  You  can 
also  save  Skype  on  your  flash  drive  and  take  it  everywhere  you  go.  It’s  just  that  convenient! 
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What  Can  Skype  Do? 


• Voice  Calls 

o Skype-to-Skype  voice  calls  (Voice  over  IP)  - free 
o Computer  to  cell  phones  and  landlines  - for  a small  fee 

• File  Transfer  - instantly  send/receive  those  large,  hard-to-send  files  safely 

• Instant  Messaging  - chat  with  any  other  Skype  user(s) 

• Video  Conferencing  - requires  a webcam 

• Call  Logs  - keep  track  of  both  calls  and  chats 

• Mobility  - log  into  Skype  from  anywhere  in  the  world 

• Skype  Universal  Translator  - up  to  28  different  languages 

• Global  User  Directory  - a phonebook  of  Skype  users  world-wide 

What  Do  You  Need  to  Use  Skype? 

Skype  is  available  for  practically  all  platforms  including  Windows,  Mac  OS  X,  and  Linux.  There  are  very  few 
system  requirements: 

• Computer  and  Internet  connection 

• Microphone  and  speakers  (or  a headset)  - for  calling 

• Webcam  - for  video  conferencing 

• With  all  three,  you  can  use  Skype  to  its  fullest  potential! 

• You  need  to  make  sure  that  your  firewall  is  not  blocking  any  of  the  Skype  functions. 

Learning  with  Skype 

Skype  presents  a great  opportunity  for  your  students  to  engage  in  active  learning  via  conference  calls  or  video 
conferences  with: 

• other  students  anywhere  in  the  world 

• you  and/or  another  teacher(s) 

• guest  speaker(s)  remotely  presenting  new  and  interesting  topics 

In  the  Virtual  Classroom 

1 . This  is  a great  communication  network  for  children  whom  are  sitting  out  because  of  a sick  day,  snow  day, 
physical  disability,  or  discipline  problem. 

2.  Skype  can  be  used  in  the  classroom  to  contact  another  classroom  anywhere  in  the  world  to  communicate 
with  pen  pals,  learn  about  new  cultures,  learn  a new  language,  and/or  share  teaching  materials  with  other 
teachers. 

Teachers  and  Students:  Using  Skype  Together 

There  are  many  great  ways  you  can  use  Skype  in  your  classroom: 

• Conduct  research  with  students  in  another  part  of  the  country  or  world 

• Practice  communication  skills  in  another  language  or  culture 

• Ask  questions  of  experts  or  community  resource  people  e.g.,  learn  about  Earth  Day  from  an  instant 
message  chat  room 

• Conduct  parent  conferences  remotely 

• Provide  mentoring  or  homework  help 

• Read,  present,  or  perform  to  a remote  audience 

• Collaborate  with  other  teachers 

• Participate  in  Professional  Development 
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Security  Features 


It  is  important  to  protect  your  students  when  they  are  using  Skype.  Students  must  be  aware  they  should  never  give 
out  personal  contact  information.  In  school  settings  extra  safety  and  privacy  considerations  are  recommended.  There 
is  a lot  of  information  on  the  Skype  website  about  staying  safe  while  using  Skype. 

What  Does  the  Research  Say? 

Skype  enhances  the  interactions  between  teachers  and  students.  Rexius  (2007)  says  “The  potential  impact  that  Skype 
technology  may  have  on  teaching  and  learning  is  significant.  It  opens  up  numerous  options  to  teacher  and  student 
interactions.  Features  that  Skype  offers  can  potentially  revolutionize  the  teaching  and  learning  process”  (p.  12). 

VanDrimmelen  (2006)  stated  “Skypecasts  have  the  potential  to  revolutionize  the  academic  community.  They  not 
only  open  up  many  options  to  teacher  and  student  interactions,  but  level  the  playing  field  in  a way  that  equalizes 
every  participant’s  voice.  They  also  create  thousands  of  new  and  exciting  possibilities  for  real-time  learning  that 
were  never  possible  before”  (f  1). 

Beside  the  convenience  of  real-time  learning,  Skype  also  provides  a state-of-the-art  technology  for  language 
learning.  Cunningham  (2004)  stated  “Education  depends  on  cutting  edge  technology,  and  with  proper  application  of 
this  new  Skype  technology  a school  can  turn  a computer  into  language  lab"  ((j[  1). 

In  Conclusion 

Skype  is  a fun  and  easy  way  for  people  to  communicate  around  the  world.  Teachers  and  students  benefit  from  the 
advanced  technology,  convenience,  and  reasonable  cost.  This  technology  has  allowed  teachers  to  implement  more 
self-paced  and  learner-centered  curriculums.  With  Skype  you  can  take  the  “distance”  out  of  distance  learning. 
Communicating  with  anyone  in  the  world,  at  little  or  no  cost,  is  now  a reality.  Using  Skype  expands  the  active 
learning  experience.  It  introduces  students  to  new  information  and  people  that  they  would  normally  never  be  able 
meet.  Skype  also  makes  learning  accessible  to  students  who  cannot  attend  class  because  of  illness,  travel,  or  family 
emergencies.  Experience  Skype  for  yourself  and  at  the  same  time  learn  how  to  broaden  the  horizons  of  your  students 
- all  for  free. 
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Overview 

In  traditional  quantitative  research  methods  that  are  based  on  algebraic  linear  models,  we  typically  obtain 
separate  measures  of  variables,  and  then  we  statistically  analyze  relations  among  measures  (e.g.,  linear,  curvilinear 
or  logistic  regression  analysis).  That  is,  we  relate  measures.  This  approach,  which  assumes  linear  and  additive 
models,  can  result  in  aggregation  aggravation — i.e.,  obfuscation  of  important  relationships  due  to  assumptions  in  the 
approach. 

In  traditional  measurement  we  aggregate  units  when  we  obtain  a value  for  a variable.  For  example,  we 
aggregate  (count  ) the  number  of  inches  when  we  measure  a person's  height,  or  we  count  the  number  of  years  when 
we  measure  someone's  age.  We  repeat  this  process  of  independent  aggregations  for  more  persons'  heights  and  ages. 
Then  we  attempt  to  do  a statistical  analysis  of  these  sets  of  independent  measures,  such  as  correlation  or  linear 
regression.  This  kind  of  thinking  stems  from  algebra — e.g.,  y = Bx  + C,  where  variable  y is  measured  separately 
from  variable  x,  and  afunctional  relationship  is  assumed  to  exist  between  x and  y,  where  B is  the  slope  and  C is  a 
constant. 

Alternatively,  we  could  measure  relations  directly.  This  is  not  a play  on  words,  but  a significant  paradigm 
change  in  conceptualizing  educational  research  problems  and  how  we  collect  and  analyze  data:  map  relations 
instead  of  measuring  variables,  and  then  analyze  relation  maps  instead  of  statistically  associating  variables.  We 
call  this  alternative  approach  MAPS  AT:  Map  & Analyze  Patterns  & Structures  Across  Time.  MAPS  AT  is  a logical 
analysis  of  relations,  not  a statistical  analysis  of  separate  measures.  In  MAPS  AT,  there  are  two  approaches  that  can 
be  taken.  In  the  Analysis  of  Patterns  in  Time  (APT)  approach,  we  map  temporal  relations.  In  the  Analysis  of 
Patterns  in  Configuration  ( APC)  approach,  we  construct  a map  of  affect-relations  in  a system. 

MAPSAT  is  a form  of  network  measurement  and  analysis.  More  specifically.  Dynamic  Bayesian  Network 
Analysis  (DBNA)  and  Social  Network  Analysis  (SNA)  are  similar  to  MAPSAT  in  that  they  are  types  of  network 
analysis  and  are  grounded  in  mathematical  digraph  theory  (Thompson,  2008;  Jensen  & Nielsen,  2007;  Brandes  & 
Erlebach,  2005).  These  three  approaches  to  network  analysis  are  more  closely  related,  compared  with  extant 
methods  of  measurement  and  regression  analysis  described  above.  While  MAPSAT  APC  methods  and  SNA  do  have 
common  aims,  the  advantages  of  MAPSAT  are  its  theory  basis  ( ATIS:  Thompson,  2006b;  2008)  and  ability  to 
measure  structural  properties  of  hypergraphs  of  multiple  sets  of  affect-relations.  Moreover,  MAPSAT  APT  methods 
differ  from  DBNA  in  that  Bayes  Theorem  is  not  assumed  nor  used  in  computing  conditional  probabilities  in  APT; 
rather  relative  frequencies  of  temporal  sequences  determine  APT  conditional  probabilities. 

Examples  of  APT 

Frick  (1990)  invented  a procedure  called  Analysis  of  Patterns  in  Time  (APT)  in  order  to  map  temporal 
relations.  Phenomena  are  observed  and  coded  with  categories  in  classifications.  The  resulting  temporal  maps  are 
then  queried  for  temporal  sequences  of  events.  For  example,  Frick  (1990)  created  temporal  maps  of  student 
engagement  and  interactive  instruction  and  found  that,  when  interactive  instruction  was  occurring,  the  temporal 
likelihood  of  student  engagement  was  very  high  (0.97).  However,  when  non-interactive  instruction  was  occurring, 
then  the  probability  of  student  engagement  was  much  less  (0.57).  Regression  analysis  of  the  same  data  (when 
engagement  and  interactive  instruction  were  aggregated  separately)  was  only  able  to  predict  32  percent  of  the 
variance  in  student  engagement — i.e.  aggregation  aggravation. 

Frick,  Chadha,  Watson  and  Zlatkovska  (2008)  used  APT  in  a study  of  teaching  and  learning  quality  in 
postsecondary  education.  Based  on  student  course  evaluations  («  = 464  in  12  different  courses),  they  found  that 
when  students  agreed  that  First  Principles  of  Instruction  occurred  in  their  courses  (Merrill,  2002)  and  they  also 
agreed  that  Academic  Learning  Time  occurred  (ALT:  Berliner,  1990;  Rangel  & Berliner,  2007),  students  were  about 
5 times  more  likely  to  be  rated  at  a High  Mastery  Level  by  their  course  instructors  than  they  were  when  they  did  not 
agree  that  First  Principles  and  ALT  occurred.  Even  more  significant,  students  were  about  26  times  more  likely  to  be 
rated  at  a Low  Mastery  Level  by  their  instructors  when  students  did  not  agree  vs.  agree  that  both  First  Principles  and 
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ALT  occurred.  When  linear  regression  analysis  of  separate  measures  of  these  variables  was  attempted,  there  was  no 
clear  or  strong  linear  relationship  between  First  Principles,  ALT  and  student  Mastery.  Less  than  10  percent  of  the 
variance  in  student  Mastery  was  predictable — i.e.  aggregation  aggravation.  On  the  other  hand,  the  triadic  patterns 
were  clear  in  the  APT  analysis. 

An  example  of  APC 

Thompson  (2006a,  2006b;  2008)  has  developed  Axiomatic  Theories  of  Intentional  Systems  (ATIS).  ATIS 
Graph  Theory  provides  a way  to  measure  17  structural  properties  of  systems  that  include  strongness,  flexibility, 
interdependence,  wholeness  and  vulnerability.  This  approach  is  called  Analysis  of  Patterns  in  Configurations 
(APC). 

A study  of  a Montessori  classroom  indicated  that  some  structural  properties  were  markedly  different  in  two 
different  types  of  learning  settings:  head  problems  and  morning  work  period.  In  the  latter,  for  example,  there  was 
much  more  interdependence  with  respect  to  affect-relation  sets  for  choice  of  learning  activities  and  guidance  of 
learning  (Koh  & Frick,  2007). 


Further  studies  that  utilize  MAPS  AT  methods 

We  describe  below  three  new  studies  that  employ  MAPSAT  APT  methods.  The  first  is  a study  of  patterns 
of  feedback  when  students  are  anonymous  or  not  when  critiquing  each  other’s  work  during  an  asynchronous 
discussion  in  an  undergraduate  course  for  preservice  teachers.  The  second  is  a study  of  the  relationships  between 
comments  students  receive  from  other  students  during  asynchronous  discussion,  and  the  quality  of  the  work  they 
offer  up  for  discussion.  The  third  study  shows  how  MAPSAT  was  used  to  analyze  sequences  of  game  player  moves 
in  the  online  Diffusion  Simulation  Game.  These  studies  further  illustrate  the  value  of  using  MAPSAT  methods  to 
prevent  ‘aggregation  aggravation’. 

Study  1:  Anonymity  to  Promote  Peer  Feedback 

Problem.  Present-day  tools  such  as  Learning  Management  Systems  (LMS)  provide  teachers  with  options 
to  make  learner-learner  communication  anonymous  (Dreher  & Maurer,  2006;  Lin,  Liu,  & Yuan,  2001;  Zhang  & 
Zhao,  2008;  Zhao,  1998).  By  eliminating  social  influence,  group  pressure,  status  and  power  differentials,  computer- 
mediated  communication  (CMC)  has  been  credited  with  bridging  social  boundaries  (Rheingold,  2000;  Walther, 

1996;  Warschauer,  2004).  However,  the  nature  of  CMC  introduces  possible  negative  consequences  of  using  these 
affordances  as  educational  tools.  Anonymity  in  CMC  poses  the  risk  of  learners  acting  less  like  individuals,  and  more 
like  an  uninhibited,  unfriendly,  and  curt  group — the  social  identity  de-individuation  effect,  SIDE  (Postmes,  Spears, 
& Lea,  1998;  Postmes,  Spears,  Sakhel,  & de  Groot,  2001).  By  combining  MAPSAT  analysis  with  computer- 
mediated  discourse  codes  (Herring,  2004)  research  questions  surrounding  the  pedagogical  decision  on  whether  or 
not  to  include  anonymity,  specifically  psuedonymity  or  type  5 anonymity  (Flinn  & Maurer,  1995;  Pfitzmann  & 
Kohntopp,  2001)  in  peer  feedback  activities  asked  (1)  to  what  extent  does  anonymity  promote  or  deter  students  from 
providing  constructive  feedback,  (2)  how  students’  comments  change  when  anonymous,  and  (3)  what  prescriptions 
can  be  made  for  teachers  who  use  wikis  and  anonymous  configurations  in  education  via  CMC? 

Method.  The  learners  were  university  education  majors  at  Indiana  University  in  Bloomington.  The  LMS 
was  an  instantiation  of  SAKAI.  Fifty  minute  feedback  sessions  were  held  where  the  asynchronous  feedback 
platform,  a wiki,  could  be  used  simultaneously  by  an  entire  class.  The  study  used  the  second  of  two  critiques  held  in 
a course.  Students  posted  a link  to  their  website  within  their  own  wiki,  and  class  members  viewed  and  posted 
comments  about  the  website  design  directly  onto  the  target  students’  wiki.  No  directions  about  the  substance  of 
comments  were  given.  Seventy-nine  of  85  students  agreed  to  have  their  work  included  in  this  study.  Thirty-seven 
(class  sizes  22  and  15)  students  made  up  the  two  anonymous  classes  where  35  of  these  students  agreed  to 
participate.  Forty-eight  students  (9,  20  and  19)  made  up  the  classes  where  both  interlocutors'  names  were  visible.  In 
this  group,  37  agreed  to  participate. 

Learners’  online  comments  were  broken  into  utterances  of  semantic  meaning  into  four  categories  to 
identify  the  substance  of  utterances.  The  mutually  exclusive  codes  were  constructive,  reactionary,  clarifying  a 
standard  and  other.  The  utterances  were  also  coded  for  tone — positive,  negative  or  neutral.  Each  utterance  received 
then  two  codes,  creating  joint  occurrences  of  the  two  categories.  In  order  to  get  a more  holistic  descriptive  statistic 
of  the  students’  participation  in  the  critique,  without  discriminating  for  style,  a critical  feedback  measure  of  both 
implicit  and  explicit  suggestions  aggregated  negative  reactions  with  constructive  utterances.  An  inter-rater 
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agreement  test  was  performed  which  met  the  requirements  set  forth  in  Fraenkel  and  Wallen  (2006).  Learner 
behavior  in  sequence  was  investigated  to  determine  which  group,  the  anonymous  or  the  known  identity,  had  the 
highest  frequency  of  patterns  resulting  in  critical  feedback. 

Results.  Between  the  anonymous  and  known  identity  groups,  the  mean  number  of  comments  was  not 
significantly  different,  but  the  mean  number  of  words  and  utterances  written  was  significantly  different. 

Anonymous  participants  wrote  significantly  more  words  (p  < .05  ) and  utterances  (p  < .01).  On  average  per  student, 
71  more  words  and  9.5  more  utterances  were  written  over  the  course  of  the  task  under  conditions  of  anonymity.  Not 
all  participants  in  the  anonymous  group  wrote  more  than  those  in  the  known  identity  group.  In  fact,  anonymous 
participants  whose  number  of  words  written  was  one  standard  deviation  below  the  mean  wrote  less  than  half  as 
many  words  as  known  identity  participants  whose  number  of  words  written  was  one  standard  deviation  above  the 
mean.  However,  the  patterns  of  the  utterance  types  were  more  revealing  of  the  differences  between  the  conditions. 

We  limited  our  analysis  of  patterns  to  sequences  of  utterances  leading  to  critical  feedback,  the  heart  of  the 
pedagogical  intervention — namely  negative  reactions  followed  by  a design  alternative  or  suggestion,  or  positive 
reactions  followed  by  critical  feedback.  Some  critical  feedback  fit  neither  pattern — for  example,  critical  feedback 
beginning  a comment.  Keeping  in  mind  that  the  anonymous  groups  had  a higher  percentage  of  critical  feedback 
overall,  means  of  frequencies  per  comment  of  both  patterns  showed  significant  differences  between  the  two 
conditions  (see  table  1).  The  frequency  of  the  positive  reactions  followed  by  critical  feedback  pattern  was 
significantly  higher  (p  < .05)  in  the  anonymous  group  as  was  the  pattern  of  a negative  reaction  followed  by  a design 
alternative  or  suggestion  (p  < .01).  Anonymous  commenters  produced  10%  more  occurrences  of  the  positive 
reaction  then  critical  feedback  pattern,  and  more  than  four  times  as  many  negative  reactions  followed  by  a design 
alternative  or  suggestion  patterns  than  the  known  identity  group.  Thus,  in  the  anonymous  condition  in  these 
critiques,  students  were  between  4 and  5 times  more  likely  to  provide  reasons  or  at  least  negative  reactions  before 
their  constructive  criticism,  and  more  likely  to  couch  their  critical  feedback  when  they  did  not  precede  it  with  a 
reason  for  change. 


Table  1.  Comment  pattern  rates  per  student  per  comment  by  group 


Comment  Pattern 
(Relative  frequency  peter 
Student  per  Comment) 

Anonymous  Group 
(n  = 35) 

Known  Identity  Group 
(n  = 37) 

Equality  of  Means 
(2-tailed  f-test) 

M 

SD 

M 

SD 

t 

df 

Positive  Reactions  then 
Critical  Feedback 

.444 

.215 

.339 

.221 

2.049* 

70 

Negative  Reaction  then 
Constructive  Utterance 

.10 

.10 

.02 

.05 

3.79** 

49 

* p < .05  **  p < .01 


Discussion.  This  APT  study  suggests  that  not  only  does  anonymity  promote,  rather  than  deter  students 
from  providing  constructive  feedback,  but  it  also  increases  the  meaningful  part  of  the  intervention,  suggesting 
changes  based  on  reasons.  Meaningful  engagement,  including  meaningful  online  interactions,  are  key  elements  for 
young  designers  to  enter  communities  of  practice  (Schwier,  Campbell,  & Kenny,  2004). The  SIDE  theory’s 
prediction  that  anonymous  interlocutors  would  be  less  inhibited  seems  to  be  evident,  while  a negative  consequence, 
curt  utterances,  were  only  found  in  an  extreme  outlier.  Students’  comments  became  less  inhibited,  however  not 
unjustified  or  random.  As  a pedagogical  prescription,  we  can  see  a basis  for  suggesting  type  5 anonymous, 
pseudonymous,  critiques  as  an  introduction  to  basic  feedback  skills,  keeping  in  mind  that  the  condition  of  anonymity 
ignores  differences  between  learners  which  might  warrant  unique  feedback  for  individuals.  The  lower  number  of 
purely  positive  reactions  in  the  anonymous  groups  suggests  they  needed  to  spend  less  time  on  typing  out 
compliments  and  spent  more  time  on  giving  suggestions.  A greater  amount  of  time  exploring  design  alternatives  and 
a lesser  amount  of  time  devoted  to  simple  politeness  may  be  a more  efficient  use  of  time  spent  on  feedback,  for  both 
the  giver  and  the  receiver.  An  anonymous  introduction  to  online  feedback  may  provide  a less  stressful  route  to 
developing  feedback  skills  in  novice  learners. 
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Study  2:  Validity  of  Computer-mediated  Formative  Peer  Assessment:  Pre-service  Teacher’s  Comments  in 
Asynchronous  CMC 

Purpose.  The  purpose  of  this  study  was  to  compare  the  patterns  of  pre-service  teachers’  online  comments 
about  peer  websites  with  instructor  ratings  of  the  same  websites.  Computer-mediated  formative  assessment  by  peers 
has  specific  pedagogical  benefits  in  the  context  of  pre-service  teacher  technology  education.  Experience  with 
formative  assessment  by  peers  prepares  pre-service  teachers  to  be  digital-age  instructors,  generates  more  feedback 
quickly  and  provides  for  more  socio-culturally  enriched  contexts. 

Theoretical  framework.  Our  understanding  of  peer  assessment  is  based  on  Topping’s  definition  that  peer 
assessment  is  “an  arrangement  in  which  individuals  consider  the  amount,  level,  value,  worth,  quality,  or  success  of 
the  products  or  outcomes  of  learning  of  peers  of  similar  status”  and  it  promotes  “cognition  and  metacognition, 
affect,  social  and  transferable  skills,  and  systemic  benefits”  (1998,  p.  197).  Peer  assessment  can  have  a 
qualitative/formative  orientation  (as  in  this  study),  quantitative/summative  orientation,  or  both.  The  systemic 
benefits  that  Topping  refers  to  include  reducing  the  assessment  burden  that  instructors  face. 

Research  questions.  We  also  needed  to  be  aware  of  our  understanding  of  the  task  being  assessed  by  the 
learners.  Learners  were  conducting  formative  assessment  on  assignments  that  attempt  to  address  a complex  and 
multi-dimensional  problem:  designing  a website  for  learning.  The  challenge  of  designing  an  instructional  website  is 
one  example  of  the  complex  (Reigeluth,  2009)  or  even  wicked  (Mishra  & Koehler,  2006)  problem  of  enhancing 
learning  experiences  with  technology.  Thus  we  conceptualized  the  study  as  an  investigation  of  how  formative 
assessment  by  peers  does  or  does  not  align  with  summative  expert  ratings,  and  how  learners  choose  to  structure  their 
discourse  with  peers. 

Method.  Five  intact  sections  («= 71)  of  a pre-service  teachers'  educational  technology  course  in  2008  at  the 
Indiana  University  Bloomington  School  of  Education  produced  two  types  of  data  for  this  study:  (a)  rudimentary 
instructional  websites,  and  (b)  comments  about  those  websites  given  by  peers.  These  data  came  from  the  same 
online  activity  described  earlier  in  the  anonymity  study.  Differences  in  number  of  participants  in  the  present  study 
were  due  to  use  of  further  data  on  the  quality  of  student  websites  according  to  instructor  ratings.  Several  student 
website  assignments  could  no  longer  be  accessed  during  data  analysis  phase  of  this  study  and  the  associated  students 
subsequently  had  to  be  dropped  from  the  current  study. 

Researchers  created  a rubric  based  on  the  assignment  guidelines  and  summatively  rated  student  websites  as 
poor,  fair,  or  excellent.  Researchers  individually  rated  each  student's  website,  compared  ratings,  and  then  achieved 
consensus  on  the  rating.  Analysis  done  in  the  anonymity  study  described  earlier  generated  data  about  student  online 
comments  about  peer  websites.  Recall  that  in  the  anonymity  study  each  online  comment  had  been  broken  into 
utterances  of  semantic  meaning  using  computer-mediated  discourse  analysis  guidelines  and  quantified  in  terms  of 
substance,  tone,  and  number  of  words  (Herring,  2001,  2004).  The  utterance  data  was  then  analyzed  to  find  the 
frequency  of  specific  utterance  patterns  associated  with  specific  website  ratings  through  the  application  of  MAPS  AT 
pattern  analysis  methods  (Frick,  1990). 

Results.  The  following  focuses  on  one  portion  of  the  results  associated  with  the  MAPSAT  analysis. 
Pearson  correlations  were  calculated  between  researcher  website  ratings  and  the  frequencies  of  two  specific 
utterance  patterns:  (1)  positive  reaction  utterance  followed  by  a constructive  utterance;  (2)  negative  reaction 
utterance  followed  by  a constructive  utterance.  Unlike  utterance  pattern  one,  the  frequency  of  the  utterance  pattern 
two  had  a significant  negative  correlation  with  the  instructor  website  rating,  r=-0.409,  n=l 1 , /?<(). 0005. 

Discussion.  The  results  of  MAPSAT  analysis  suggest  a possible  issue  with  the  use  of  formative  peer 
assessment  that  would  be  difficult  to  detect  with  other  methods.  Pattern  one  begins  with  a positive  reaction  that  may 
lessen  the  negative  emotional  impact  of  the  constructive  utterance.  Pattern  two  makes  no  attempt  to  soften  the 
possible  negative  emotional  impact  of  the  constructive  utterance.  Pattern  two  is  harsher  than  pattern  one  since  it 
starts  with  a negative  reaction  instead  of  a positive  reaction.  The  results  show  that  the  lower  the  website  rating  the 
higher  the  frequency  of  the  harsher  pattern  two.  The  affective  or  emotional  impacts  of  receiving  overly  negative  or 
harsh  feedback  from  peers  on  learning  are  not  well  understood  (Picard  et  al.,  2004)  but  teacher  feedback  patterns, 
such  as  the  “feedback  sandwich”  (Eckstein  et  al.,  2002),  exist  to  reduce  the  negative  emotional  impact  of  providing 
too  much  critical  feedback.  Those  with  poor  quality  websites  did  not  receive  significantly  more  critical  feedback 
that  was  couched  with  a positive  reaction  than  did  those  with  higher  quality  websites.  This  finding  suggests  that  the 
preservice  teachers  in  this  study  were  harsh  since  they  did  not  encourage  poorer  performing  learners  as  an 
experienced  teacher  might. 

Conclusions.  MAPSAT  analysis  enabled  the  researchers  to  analyze  the  structure  of  student  comments  in  a 
way  that  accounts  for  the  temporal  dimension.  In  the  context  of  formative  peer  assessment,  what  is  written  when 
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matters.  This  study  found  significant  correlations  between  specific  patterns  of  pre-service  teachers’  online 
comments  about  peer  websites  and  instructor  ratings  of  the  same  websites. 

This  study  has  important  implications  for  both  teacher  educators  and  educational  technology  researchers. 
Most  Schools  of  Education  devote  at  least  one  course  to  the  topic  of  educational  technology  (Moursund.  1999)  and 
consequently  there  is  a need  to  provide  pre-service  teachers  with  sufficient  formative  feedback  on  the  digital 
artifacts  that  they  create  during  these  classes.  Distributing  the  task  of  providing  feedback  to  students  may  enable 
technology-savvy  peers  to  provide  feedback  that  instructors  may  not  be  able  to  give  due  to  time  constraints-.  In 
addition,  lack  of  sufficient  and  timely  feedback  in  online  classes  may  contribute  to  the  disconnected  feeling  that 
many  students  experience  in  online  classes  (Ko  & Rossen,  2001).  Peer  assessment  could  help  to  mitigate  feelings  of 
disconnection  by  providing  increased  opportunities  for  both  receiving  timely  feedback  and  for  engaging  with 
classmates. 

Future  research  could  focus  on  what  patterns  of  feedback  align  with  guidelines  regarding  what  type  of 
feedback  has  the  most  impact  on  learning  (Shute,  2008)  or  on  the  relationship  between  the  feedback  that  a student 
creates  and  the  quality  of  the  products  they  create. 

Study  3:  An  Analysis  of  Patterns  of  Gameplay  Data  in  the  Diffusion  Simulation  Game 

This  study  describes  how  the  MAPSAT  method  Analysis  of  Patterns  in  Time  (APT)  was  used  to  analyze 
gameplay  data  from  the  online  Diffusion  Simulation  Game  (DSG).  The  DSG  was  originally  developed  as  a board 
game  by  Dr.  Michael  Molenda  and  Patricia  Young  in  1976  to  help  Instructional  Systems  Technology  graduate 
students  at  Indiana  University  learn  the  theory  of  diffusion  of  innovations.  An  online  version  of  the  DSG  was 
developed  in  2002  for  use  in  the  department’s  growing  distance  education  program.  The  use  of  APT  to  analyze  the 
data  led  to  interesting  findings  that  would  not  have  been  revealed  using  traditional  statistical  methods.  These 
findings  include  differences  between  the  strategies  used  by  expert  and  non-expert  players  as  indicated  by  the  most 
common  sequences  of  activities  selected,  as  well  as  the  most  common  joint  occurrences  of  individual  staff  members 
selected  to  engage  in  particular  activities. 

Statement  of  the  problem.  Fidelity  is  the  degree  to  which  a simulation  is  faithful  to  that  which  it 
simulates.  While  high  fidelity  may  seem  desirable,  several  researchers  have  found  that  it  is  not  always  necessary  and 
may  in  fact  deter  learning  (Alessi  & Trollip,  2000;  Feinstein  & Cannon,  2002;  Winn,  2002).  Reigeluth  and  Schwartz 
(1989)  theorized  that  the  most  fundamental  aspects  of  a simulation  should  have  high  fidelity,  while  lower  fidelity  is 
appropriate  for  the  more  superficial  aspects  that  may  otherwise  lead  to  cognitive  overload  and  impede  learning  and 
transfer. 

In  the  DSG,  the  strategies  used  are  the  most  fundamental  concepts  needed  for  learners  to  understand  how 
to  take  on  the  role  of  a change  agent  effectively.  However,  the  strategies  that  have  been  successful  in  the  game  have 
never  been  assessed  in  terms  of  their  fidelity  to  the  strategies  predicted  to  be  effective  by  the  theory  of  the  diffusion 
of  innovations. 

The  theory  of  diffusion  of  innovations  attempts  to  explain  how  innovations  are  adopted  by  a group  of 
people  (Rogers,  2003).  The  theory  offers  strategies  that  a can  be  used  by  a change  agent — a person  or  group  that 
wishes  to  promote  the  adoption  of  an  innovation — in  order  to  increase  the  probability  that  the  innovation  will  be 
adopted  as  well  as  to  speed  up  the  adoption  process.  The  purpose  of  this  study  was  to  examine  the  fidelity  of  the 
DSG  to  the  theory  of  innovations  by  determining  whether  the  strategies  that  are  successful  in  the  DSG  are  congruent 
with  the  strategies  that  the  theory  would  predict  to  be  effective. 

What  is  the  Diffusion  Simulation  Game  (DSG)?  The  DSG  is  a simulation  game  in  which  a player  takes 
on  the  role  of  a change  agent  in  a junior  high  school.  The  main  learning  goal  of  the  game  is  the  comprehension  of 
the  theory  of  the  diffusion  of  innovations  first  proposed  by  Rogers  in  1962.  The  player’s  objective  in  the  game  is  to 
persuade  as  many  of  the  22  staff  members  as  possible  to  adopt  an  innovation-peer  tutoring.  To  be  effective,  players 
must  learn  appropriate  application  and  sequencing  of  the  available  diffusion  strategies. 

The  primary  game  mechanic  of  the  DSG  is  the  selection  of  diffusion  activities  that  the  player,  as  the  change 
agent,  can  choose  in  order  to  persuade  staff  members  to  become  adopters  of  the  innovation.  The  diffusion  activities 
(Talk  to.  Ask  Help,  Pilot  Test,  Site  Visit,  Print,  Presentation,  Demonstration,  Self-administered  Workshop, 
Professional  Workshop,  Workshop  to  Develop  Materials,  Focal  Mass  Media,  Compulsion,  and  Confrontation)  each 
cost  the  change  agent  a different  number  of  weeks  to  complete  and  have  varying  impact  on  staff  at  different  times  in 
the  game.  With  only  a two-year  calendar  for  the  change  agent  to  complete  the  diffusion  process,  selecting  the  most 
efficient  diffusion  activities  at  the  appropriate  points  in  the  game  is  crucial  to  winning  the  game. 

Players  also  have  access  to  information  on  individual  staff  members  and  diagrams  of  the  interpersonal 
communication  channels  that  exist  (lunch  mates,  committees,  and  social  networks).  This  information  can  be  used  to 
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determine  which  staff  members  to  include  in  the  selected  diffusion  activities.  While  some  staff  members  are  well 
connected  and  influential,  others  are  not.  Therefore,  the  selection  of  staff  for  the  activities  is  as  important  in  the 
strategy  of  the  game  as  the  selection  of  activity. 
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Staff  Members 


STAFF 

MEMBER 

INFORMATION 

AWARE- 

NESS 

INTEREST 

TRIAL/ 

APPRAISAL 

ADOP- 

TION 

A 

Principal 

Very  ambitious  (has  a 20-year  plan);  memoer  of  the  Rotary  Club 
and  local  Republican  Club  (active  in  both):  delegates  authority  to 
able  administrative  assistants  and  runs  a "tight  ship.  Has  a 
"masteTs-olus"  in  administration. 

■ 

■■■■■ 

■■■■■ 

0 

B 

Secretary 

Has  been  in  this  school  sinoe  it  was  built  and  quite  indispensable 
to  its  smooth  functioning.  Runs  most  faculty  social  functions. 

l 

■1 

C 

Janitor 

Fond  of  children,  but  stern.  He  tends  to  allow  extensive  use  of  the 
school  building,  but  h8s  strict  rules  and  is  inflexible  about 
infractions. 

D 

Math  Chairman 

A veteran  in  the  school,  he  runs  the  most  experienced  department 
with  a minimum  of  effort.  Is  involved  much  more  in  out-of-school 
activities  such  as  the  local  garden  dub  and  conservation 
organization.  Still  regrets  the  receal  of  prohibition. 

□ 

E 

Math  Teacher 

Working  hard  to  oomplete  her  Master's  thesis  because  the  salary 
increment  will  supplement  husband's  unoertain  income  from  new 
construction  business.  Acts  as  attendance  coordinator  and  has 
tended  to  get  somewhat  sour  about  students  and  their  parents 

□ 

F 

Math  Teacher 

Just  about  the  most  respected  and  liked  teacher  in  the  school. 
Students  enjoy  the  humorous  examples  he  uses  in  teaching 
algebra.  Exudes  a sense  of  self-confidenoe  and  has  no  enemies 
among  the  staff  Serves  as  advisor  to  the  Student  Coundl.  Never 
misses  a PTA  meeting. 

■ 

■■■ 

■■■ 

0 

G 

Science  Chairman 

Known  more  for  his  eagerness  and  energy  than  administrative 
skill.  He  comes  up  with  new  instructional  ideas  faster  than  they 
can  be  implemented  sinoe  he  is  working  on  a Master  s and  often 
tries  out  suggestions  discussed  in  his  graduate  classes.  Among 
older  staff,  he's  oonsidered  somewhat  erratic. 

1 

II 

II 

0 

H 

Science  Teacher 

Has  spent  years  collecting  specimens,  preparing  new  instructional 
materials,  and  organizing  laboratory  procedures.  Open  to  new 
techniques  of  proven  value.  Insists  on  an  orderly  classroom.  His 
success  in  teaching  makes  his  reseeded  by  alumni  and  fellow 
teachers. 

l 

■■■■■ 

■1 

□ 

STAFF 

MEMBER 

INFORMATION 

AWARE- 

NESS 

INTEREST 

TRIAL/ 

APPRAISAL 

1 

Science  Teacher 

□ 

J 

Social  Studies  Chairwoman 

Although  competent  and  knowledgeable,  she  is  head  of  a 
somewhat  troubled  department  charaderized  by  a high  turnover 
rate.  Most  of  the  work,  administrative  and  pedagogical,  has  to  be 
done  singlehandedly  by  her.  Working  on  a Master's  degree  in 
American  Studies. 

□ 

K 

Social  Studies  Teacher 

□ 

L 

Social  Studies  Teacher 

An  enthusiastic  first-year  teacher.  She  defines  education  as  total 
personal  growth  and  as  much  is  equally  involved  in  her  classroom 

■ 

■1 

■1 

0 

^lttpT/wwwundiana^edu/Mstdemc^dsgaecV^ 


COST 

INFORMATION 

1 

week 

Personal:  Find  out  what  sort  of  people  the  staff  members 
are 

1 

week 

Lunchmates:  Observe  carefullvto  see  who  lunches  with 
whom  each  noon 

1 

week 

Committees:  Find  out  who  are  members  of  the  various 
formal  committees  set  up  in  the  school 

2 

weeks 

Social:  Observe  the  out-of-school  social  patterns  to  learn 
who  plays  poker  together  who  bowls  together  etc 

Diffusion  Activities 

COST 

DIFFUSION  ACTIVITY 

1 

week 

Talk  To:  You  make  a conscious  effort  over  a period  of  about 
one  week,  to  engage  any  ONE  person  in  a number  of  one  to 
one  conversations 

1 

week 

Ask  Help:  You  ask  any  ONE  of  the  staff  for  advice  or  for  help 
in  one  of  your  projects  preparing  some  learning  materials 
setting  up  a demonstration  running  a workshop  etc 

2 

weeks 

Pilot  Test:  You  attempt  to  influence  ONE  teacher  by  asking 
to  let  you  conduct  an  informal  pilot  test  of  peer  tutoring  with 
his/her  students 

4 

weeks 

Site  Visit:  You  select  anv  FIVE  persons  to  visit  Lighthouse 
School,  in  the  next  state,  where  an  exemplary  tutoring 
program  is  in  progress 

1 

week 

Print:  You  circulate  a brochure  describing  the  many 
advantages  of  peer  tutoring  to  any  FIVE  persons 

3 

weeks 

Presentation:  You  aet  on  the  aaenda  of  a regularlv 
scheduled  staff  meeting  to  explain  about  peer  tutoring  and 
encourage  discussion  about  it 

3 

weeks 

Demonstration:  You  invite  the  staff  into  a particular 
teacher's  classroom  (an  adopters!)  to  see  peer  tutoring  in 
action 

5 

weeks 

Traininq  Workshop  (Self):  You  conduct  an  in-service 
workshop  which  trains  teachers  in  the  operational  details  of 
setting  up  and  carrying  on  a peer  tutoring  program  in  their 
classrooms 

2 

weeks 

Trainina  Workshop  (Profl:  You  arranae  to  have  Professor 
Portney  of  Centralia  Teachers  college  conduct  an  in-service 
workshop  on  "Peer  Tutoring:  Its  Role  in  Student 
Self-Development " 

5 

weeks 

Materials  Workshon:  You  conduct  an  inservice  workshop  in 
which  teachers  team  up  to  develop  creative  materials-games 
flash-cards  etc  fnr  student  tutor  use 

Figure  1.  Screen  capture  of  the  Diffusion  Simulation  Game  (vl)  in  progress. 


To  get  staff  members  to  become  adopters,  the  player  must  progress  the  staff  members  through  the  adoption 
phases  of  awareness,  interest,  and  trial.  For  each  staff  member,  a variable  number  of  boxes  exist  for  each  of  these 
phases  that  represent  how  far  along  the  staff  member  is  in  the  innovation  decision  process.  All  staff  members  have 
one  box  that  must  first  be  filled  in  the  awareness  phase,  two  to  seven  boxes  in  the  interest  phase,  and  two  to  seven 
boxes  in  the  trial  phase.  The  number  of  boxes  can  give  players  insight  as  to  what  adopter  type  each  staff  member  is: 
innovator,  early  adopter,  early  majority,  late  majority,  or  laggard. 
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Research  questions.  As  previously  stated,  the  purpose  of  this  study  was  to  examine  the  fidelity  of  the 
DSG  to  the  theory  of  the  diffusion  of  innovations  by  determining  whether  the  strategies  that  are  successful  in  the 
DSG  are  congruent  with  the  strategies  that  the  theory  would  predict  to  be  effective.  Specifically,  the  researchers 
attempted  to  answer  the  following  questions: 

1 . What  strategies  were  successful  in  the  game  sessions  being  studied? 

2.  Are  the  strategies  that  are  successful  within  the  game  aligned  with  strategies  that  would  be  predicted  by  the 

theory  of  the  diffusion  of  innovations? 

3.  Do  other  successful  game  strategies  exist  that  do  not  relate  to  the  theory  of  the  diffusion  of  innovations? 

Data  collection.  Due  to  a growing  interest  in  the  game  outside  of  Indiana  University,  a free,  limited 

version  was  developed  in  2006  which  required  no  university  login.  The  data  from  this  study  was  generated  from  the 
first  10,000  game  sessions  of  the  free  version  occurring  between  Oct.  7,  2006  and  April  4,  2009.  Of  the  10,000 
game  sessions,  almost  half  (4,489  games)  ended  with  the  player  never  selecting  a diffusion  strategy.  The  player 
simply  did  not  attempt  to  play  the  game.  The  data  set  was  further  narrowed  down  to  the  game  sessions  that  would 
give  enough  information  about  what  strategies,  if  any,  were  being  used  throughout  the  entire  game.  Therefore  the 
final  data  set  used  for  the  study  consisted  of  2,361  finished  games — all  game  play  sessions  in  which  players 
completed  all  72  weeks  or  won  the  game  by  convincing  all  22  staff  members  to  adopt  the  innovation.  The  sample  of 
players  was  unknown  as  anybody  with  Internet  access  may  have  played  the  game.  However — given  how  knowledge 
of  the  game  was  spread,  communication  from  those  using  the  game,  and  IP  address/login  information — it  is  likely 
that  many  players  were  students  outside  of  Indiana  University  who  were  learning  about  the  diffusion  of  innovations 
in  an  educational  setting. 

Data  analysis  and  results.  The  data  set  was  divided  into  three  categories  to  be  used  as  a means  of 
comparing  strategies  used  in  the  game.  Successful  strategies  represented  the  341  game  sessions  in  which  the  player 
got  100%  of  the  22  staff  members  to  adopt  the  innovation.  Though  all  of  these  strategies  led  to  triumph,  the 
efficiency  of  how  quickly  players  won  the  game  varied  greatly.  While  some  players  needed  all  72  weeks  of  the 
game  to  win,  others  needed  fewer  than  40.  The  next  category  consisted  of  the  488  games  in  which  16  to  21  adopters 
were  gained.  Due  to  an  element  of  randomness  designed  into  the  DSG,  these  strategies  may  have  been  successful  or 
unsuccessful.  Unsuccessful  strategies  represented  the  remaining  1,532  games  in  which  fewer  than  16  staff  members 
became  adopters. 

Not  only  was  there  summative  information  on  each  of  the  2,361  finished  games,  information  for  each  of  the 
107,294  turns  taken  within  games  was  available.  Descriptive  and  inferential  statistics  were  used  to  understand  the 
data  using  each  game  session  as  the  unit  of  analysis.  Space  precludes  a detailed  discussion  of  the  results.  Therefore, 
it  must  suffice  to  say  that  for  the  eight  strategies  examined,  which  were  inferred  from  the  diffusion  of  innovations 
theory,  f-tests  found  statistically  significant  differences  between  games  with  successful  strategies  and  games  with 
unsuccessful  strategies  for  seven  of  the  identified  strategies.  This  is  not  surprising  given  the  large  sample  size 
(Enfield,  Myers,  & Lara,  2009).  However,  to  answer  the  research  questions  of  the  study,  analysis  of  the  strategies 
within  each  game  was  needed.  APT  provided  methods  for  analyzing  both  the  sequences  of  activities  selected  and 
the  joint  occurrences  of  individual  staff  members  and  activities  selected  within  games. 

To  understand  which  sequences  of  activities  were  used  within  games,  a frequency  count  for  every  sequence 
of  activities  occurring  in  the  data  for  combinations  of  2,  3,  and  4 turns  was  calculated.  The  analysis  of  sequences  led 
to  the  discovery  of  strategies  that  could  not  have  been  found  using  inferential  statistics.  For  instance,  the  most 
common  sequence  of  four  activities  in  games  with  successful  strategies  was  the  use  of  the  print  activity  four  times  in 
a row,  while  in  games  with  poor  strategies  it  was  the  use  of  the  talk  to  activity  four  times  in  a row.  Furthermore, 
APT  also  provided  methods  for  looking  at  the  joint  occurrences  of  the  activities  selected  and  the  staff  members 
selected  for  each  activity.  In  games  with  successful  strategies,  the  print  activity  was  3.8  times  more  likely  to  be  used 
with  the  principal,  3.3  times  more  likely  to  be  used  with  opinion  leaders,  and  3.5  times  more  likely  to  be  used  with 
highly  connected  staff  members  (those  with  connections  to  10  or  more  other  staff  members)  than  in  games  with  poor 
strategies;  these  targeting  strategies  are  predicted  by  Rogers’  theory  to  be  successful.  In  examining  these  joint 
occurrences  it  becomes  clear  that  successful  strategies  utilize  communication  channels  (the  print  and  local  mass 
media  activities)  with  influential  stakeholders  (the  principal,  opinion  leaders,  and  staff  members  with  large  social 
networks). 
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Summary 


These  three  empirical  studies  have  illustrated  the  value  of  a MAPSAT  approach  to  analysis  of  relations. 
Relations  were  viewed  as  temporal  patterns  in  these  studies.  Instead  of  aggregating  values  for  each  unit  of  analysis 
separately  and  then  using  a linear  model  to  study  the  relationships  among  variables,  patterns  within  each  unit  of 
analysis  were  identified  and  counted.  Thus,  the  “measure”  of  a variable  was  derived  by  counting  instances  of  a 
pattern  or  relation  itself  within  each  case  (unit  of  analysis). 

For  example,  in  Study  1,  a pattern  was  identified:  an  utterance  which  was  a negative  reaction  to  a peer’s 
website,  followed  by  an  utterance  that  was  a constructive  comment  (e.g.,  Yuk!  Hard  to  read,  [negative  reaction]  You 
need  greater  contrast  between  text  and  the  background  to  improve  legibility  [constructive  comment]).  For  each 
student  (the  unit  of  analysis),  this  pattern  was  aggregated  so  that  a rate  of  occurrences  of  such  an  utterance  pattern 
per  comment  made  by  that  student  could  be  calculated.  Thus,  the  pattern  rate  was  the  measure  which  became  the 
value  of  the  variable  (negative  reaction  then  constructive  utterance).  The  variable  was  conceived  as  a pattern  within 
the  conceptual  framework  of  the  research.  Then  these  variable  values  were  averaged  across  students  within  each 
group  (anonymous  vs.  known  identity)  and  a standard  parametric  statistical  procedure  was  used  to  compare  groups 
(a  f-test  here). 

What  is  important  to  note,  however,  is  that  if  these  utterance  types  were  aggregated  separately,  where  one 
variable  is  ‘negative  reaction  utterance’  and  the  other  variable  is  ‘constructive  comment  utterance’,  then  we  would 
have  two  variables  not  one.  The  only  way  to  look  at  the  relation  between  those  two  variables,  when  aggregated 
separately,  is  some  kind  of  statistical  correlation  or  covariance  analysis.  This  can  obfuscate  the  actual  temporal 
relation.  In  fact,  it  is  mathematically  impossible  to  determine  joint  or  sequential  frequencies  when  only  marginal 
distributions  are  known,  except  when  relations  are  deterministic  (cf.  Frick,  1990).  This  is  what  is  meant  by 
“aggregation  aggravation.”  If  data  are  collected  such  that  the  patterns  are  preserved,  then  temporal  or  structural 
mapping  can  subsequently  be  done  as  “measures”  of  the  values  of  variables.  If  not,  then  MAPSAT  cannot  be  done 
at  all,  or  can  only  be  done  in  a limited  fashion  as  in  Frick,  Chada,  Watson  and  Zlatkovska  (2008).  Like  Humpty 
Dumpty  who  fell  off  the  wall  and  broke  into  pieces,  the  relationships  among  the  pieces  when  Humpty  Dumpty  was 
whole  are  lost.  Thus,  advance  planning  of  studies  so  that  MAPSAT  methods  can  be  used  is  paramount. 
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Abstract 

This  study  is  a Category  2:  Work  in  Progress  Report  which  delineates  the  progress  of  a study 
currently  underway.  Participants  of  this  study  included  54  preservice  teachers  in  their  senior  year 
of  an  elementary  teacher  education  licensure  program  at  a Southwestern  United  States  university. 

Initial  findings  reveal  mixed  response  to  integration  of  Quest  Atlantis  into  a university  reading 
methods  class.  Responses  varied  primarily  along  age  group  and  familiarity  with  technology.  Data 
revealed  a connection  between  the  number  of  participant  logins  and  positive  participant  attitudes 
about  playing  in  a virtual  space.  This  study  revealed  the  need  to  build  a substantive  background 
and  link  between  theory  and  practice  for  preservice  teachers. 

We  assign  copyright  of  this  manuscript  to  AECT  and  the  Research  and  Theory  Division  of  AECT. 

Introduction 

It  is  our  theory  that,  in  an  attempt  to  prepare  teachers  for  classrooms  of  tomorrow  to  create  dynamic  literacy 
environments,  teacher  preparation  programs  should  be  moving  technology  instruction  beyond  word  processing, 
spreadsheets  and  use  of  the  internet.  In  this  presentation  we  will  explore  the  theoretical  link  between  the  use  of 
Avatars  (i.e.  alternate  identities  found  in  3D  virtual  environments  and  video  games)  to  move  students  toward 
specific  state  mandated  learning  goals.  This  paper  applies,  through  the  use  of  interdisciplinary  studies,  a new 
paradigm  of  what  it  means  to  teach  and  learn.  The  use  of  Avitars  to  support  classroom  instruction  must  first  begin 
with  introducing  the  theoretical  framework  of  virtual  literacy  practices  to  both  in-service  and  preservice  teachers  in 
order  to  allow  educators  to  understand,  through  actual  play,  the  cognitive  and  pedagogical  connections  between 
what  is  learned  in  a virtual  game  and  traditional  classrooms.  We  will  first  review  long  standing  theories  on  the  use  of 
play  as  a cognitive  tool  for  learning,  then  explore  the  connection  play  and  cognition. 

Related  Literature 

Make  Believe  as  an  Instructional  Tool.  For  over  a century,  educational  theorists  (Goldman,  1998;  Mead, 
1934/2002;  Piaget,  1962;  Spodek,  1973)  emphasized  the  importance  of  play  as  a cognitive  tool  for  learning.  Mead 
(1934/2002)  noted  that,  in  relation  to  building  a socially  situated  identity,  there  is  a strong  need  for  children  to  play 
with  peers  to  develop  the  idea  of  self  in  relationship  to  others.  Additionally,  Vygotsky  (1978)  noted  the  social 
significance  of  play  as  a way  to  rehearse  adult  behavior.  Goldman,  (1998)  found  that,  for  young  children,  social  play 
provides  an  important  and  unique  context  within  which  cognitive  and  social  skills  may  be  acquired  and  practiced. 

As  a component  of  effective  learning,  little  has  been  written  of  the  educational  potential  of  video  games 
and  simulations  as  a form  of  play  for  all  age  groups  through  the  use  of  opportunities  for  social  and  educational  play 
activities.  As  Piaget  (1978)  notes,  “symbolic  play,  then,  is  only  one  form  of  thought,  liked  to  all  of  the  others  by  its 
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mechanism,  but  having  as  its  sole  aim,  satisfaction  of  the  ego,  i.e.  individual  truth  as  opposed  to  collective  and 
impersonal  truth,  but  we  are  still  faced  by  the  question  in  make  believe”  (p.  167).  Make  believe  then  may  lead  to  a 
sense  of  personal  choice. 

Psychological  Nature  of  Personal  choice.  Across  many  domains  of  inquiry,  psychologists  have  discussed 
the  significance  of  choice  as  a beneficial  factor  for  increasing  an  individual’s  sense  of  personal  control  (Rotter, 

1966;  Taylor,  1989)  as  well  as  intrinsic  motivation  as  a method  through  which  humans  engage  with  the  world 
(deCharms,  1968;  Deci,  1981).  While  personal  choice  is  a highly  motivating  factor,  within  a highly  structured 
environment,  choice  may  be  limited.  Having  the  ability  to  play  with  one’s  identity  then  becomes  the  means  through 
which  mediated  learning  may  occur. 

Pedagogical  aspects  of  role  play.  According  to  Yellin,  Jones,  and  Devries  (2008),  dramatic  play  provides 
for  opportunities  for  cooperative  learning,  social  skills  and  verbal  abilities.  Role  playing  and  theatre  arts  have  been 
used  to  support  learning  and  “transmediation”  in  the  Social  Studies  (Wallace,  2003)  and  Language  Arts.  High 
quality  inquiry  and  play  rich  environments  support  increased  memory,  support  language  acquisition,  and  literacy 
(Berghoff,  Egawa,  Harste,  Hoonan,  2000).  Within  video  games  and  simulations,  using  first  person  perspective 
allows  learners  to  align  with  an  affinity  group,  practice  skills  and  engage  with  problem  solving  in  a relatively  risk 
free  environment  while  engaging  in  meaningful  learning  (Gee,  2008;  Zieehezarjeribi,  Graves,  Gentry,  2010). 

Psychological  nature  of  Avatars.  In  research,  specifically  targeted  to  the  psychological  impact  of  Avatars  in 
video  games,  Lim  and  Reeves  (2005)  found  that,  through  the  use  of  Avatars,  “players  become  emotionally  and 
psychologically  involved  in  the  game”  and  children  learn  to  negotiate  fictional  world  through  symbolic 
representation.  This  symbolic  representation  mediates  language,  cultural,  age,  race  and  gender  difference. 
Additionally,  Bonk  (2009)  discusses  the  advantages  of  using  virtual  worlds  as  part  of  blended  learning  at  the 
university  level  when  he  makes  a formative  claim  that  “the  synchronous  or  real  time  interactions  in  Second  Life 
provide  a sense  of  a personal  touch  or  presence  that  is  not  felt  with  online  discussion  forums  and  blogs.  Synchronous 
technologies  encourage  immediate  interaction,  collaboration,  and  information  on  demand  that  connects  students 
more  deeply  to  a course”  (p.  280). 

Due  to  the  psychological  nature  of  the  learning  which  occurs  through  anonymity  we  sought  to  determine 
first,  how  and  why  teachers  would  choose  to  incorporate  virtual  spaces  in  their  own  classrooms.  The  first  step  in  the 
process  of  creating  dynamic  learning  environments  for  elementary  students  had  to  first  begin  with  determining  what 
future  teachers  thought  of  virtual  spaces. 


Methods 

Background  and  Setting.  First  year  participants  of  this  study  included  54  preservice  teachers  in  their  senior 
year  of  an  elementary  teacher  education  licensure  program  at  a Southwestern  United  States  university.  The 
population  was  homogeneous  and  consisted  primarily  of  young  White  females.  The  three  males  in  the  population 
were  all  White,  two  of  the  female  participants  self-identified  as  “African-American”  and  “Hispanic.”  The  course 
description  included  “methods  of  assessment  and  instruction  for  strategy  building,  comprehension,  vocabulary,  word 
identification.  . .addressing  reading  differences  including  diverse  learner  reading  processes  and  development  of 
literacy  in  English  or  ELL.”  As  a routine  component  of  the  course,  participants  were  asked  to  complete  at  least  5 
missions  (Table  1)  found  inside  a virtual  game  entitled  Quest  Atlantis  (Barab,  et.al,  2005).  Students  were  given 
points  for  each  mission  completed.  Mission  completion  ranged  from  6 to  45  “logins”  depending  on  the  participant. 
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Table  1.  Required  Quest  Atlantis  Missions 


Mission 

Content 

PD  1 -Intro  Mission 

Introduces  players  to  the  QA  context.  Players  learn  how  to  navigate 
through  the  virtual  space 

Statistics:  Which 
Brand  of  Bike? 

Mayor  Enoch  solicits  help  from  players  as  consultants  in  his 
upcoming  election.  Players  must  learn  to  collect  and  interpret 
statistical  data  and  create  a report  for  the  mayor. 

Spacenik:  NASA 
Needs  You 

Players  attend  a NASA  conference  and  collect  information  about  an 
asteroid  named  Atlas  which  is  on  a trajectory  for  Earth.  Players  must 
have  two  different  scientific  perspectives,  report  back  to  Mr. 
Newman,  correct  misunderstandings  about  a scientific  concept,  and 
travels  in  a rocket. 

Taiga  (Pre-Mission): 
Abby's  Aquarium 

Players  are  introduced  to  eutrophication,  using  water  quality 
analysis,  by  helping  to  maintain  Abby's  fish  tanks  and  talking  to 
Ranger  Bartle.  Players  must  learn  to  balance  the  water  quality  in  a 
fish  tank,  read  Ranger  Battle's  email,  and  discuss  the  mission  in 
class. 

Cinder  Creek: 
Biological  Indicators 

Players  learn  to  measure  "biological  indicators  of  water  quality."  by 
using  a microscope  to  identify  "macroinvertebrates"  in  a water 
sample,  analyze  data,  and  create  a report  on  the  player’s  findings. 

Mesa  Verde  OLD: 
Welcome  to  Mesa 
Verde 

Players  help  a friend  who  is  having  bad  dreams  about  a missing 
puzzle  piece  in  a stone  image.  Players  are  asked  to  correct 
misconceptions  about  the  “discovery”  of  America.  Players  see  Mesa 
Verde  through  the  lens  of  an  archeologist  by  collecting  surveys 
measurements,  talking  to  park  rangers  and  tourists. 

At  the  end  of  the  semester,  students  had  the  option  of  submitting  answers  to  an  informal  online  interview.  These 
digital  responses  became  the  foundation  for  Discourse  Analysis  regarding  positive  or  negative  experiences  with  the 
virtual  experience. 

Quest  Atlantis  (QA).  In  order  to  support  comprehension,  participants  were  first  introduced  to  the  back  story  of  the 
virtual  world.  Participants  first  viewed  the  QA  legacy  video  and  were  provided  with  login  numbers  which  became 
their  virtual  identity.  After  completing  several  tasks,  participants  were  allowed  to  modify  their  Avatars  (Fig  1.). 
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While  the  Avatar  options  were  limited,  many  of  the  participants  were  excited  to  have  the  ability  to  modify 
their  attire  as  their  mood  changed.  As  participants  received  feedback  or  after  “missions”  were  successfully 
completed,  participants  were  addressed  using  only  their  number,  “Fantastic  data  analysis  on  the  Speedy  Spokes, 
Thirty-One.  The  council  appreciates  the  knowledge  you  are  providing  . . . . “ In  QA,  Avatars  then  provide 
anonymity  while  participants  learn  to  learn  inside  a virtual  space. 

Data  Collection  and  Analysis.  Data  consisted  of  written  responses  to  classroom  instruction  and  classroom 
discussions  found  on  an  asynchronous  discussion  board  (BlackBoard)  woven  into  normal  classroom  instruction. 
These  responses  were  analyzed  using  Critical  Discourse  Analysis  (Fairclough,  2003;  Rogers,  2003).  Participants 
were  required  to  follow  a discussion  board  rubric  which  included  a minimum  of  500  words  for  full  credit.  Due  to  the 
asynchronous  nature  of  the  discussion  board,  individual  participant’s  comments  were  available  for  viewing  and 
feedback  from  all  class  members. 


Data  Analysis 

Initial  data  analysis  reveals  a mixed  response  to  learning  to  play  in  a “third  space”  environment.  A 
significant  difference  resided  between  generations  of  students.  For  instance,  preservice  teachers  within  the  age  range 
of  35-40  struggled  to  learn  the  virtual  world  but  found  respite  in  the  “virtual  homework”  assignments  required  for 
the  course,  while  students  within  the  age  range  of  19-35  were  quick  to  adapt  their  limited  experience  with  video 
games  to  their  comfort  with  the  use  of  technology. 

In  some  cases,  the  pre-service  teacher  players  remained  focused  how  they  were  represented  through  their 
Avatar  in  the  virtual  world  than  accuracy  of  their  statistical  reports  and  analysis  of  eutrophication  in  a fish  pond. 
Several  participants  gendered  themselves  differently  than  they  did  in  the  “real”  world  (2  older  females  dressed  their 
Avatars  as  young  males).  As  one  of  the  generally  self-marginalized  male  participants  stated,  “I  like  playing  with 
other  (university  students)  in  the  game  (Figure  2).  I stand  at  the  entrance  to  the  world  and  tell  them,  welcome  to  my 
world.” 


Figure  2.  Avatar  Welcoming  Participants  into  Quest 
Atlantis 


The  ability  to  take  on  a new  expressive  identity  underscores  the  potential  impact  of  the  use  of  virtual 
worlds  for  building  social  skills  in  a low  risk  environment.  Within  the  virtual  space,  Avitars  do  not  hesitate  to  ask  for 
help  and  are  more  apt  to  advocate  for  themselves.  Several  students  explained  that,  “to  ask  for  help  in  (the  game)  is 
no  big  deal  but  when  we  are  in  class  we  don’t  want  to  sound  stupid.” 

Virtual  vs  face-to-face  participation.  Expressing  their  feelings  about  the  traditional  classroom  activities, 
some  students  made  the  point  that  the  anonymity  of  an  Avatar  allows  a certain  amount  of  freedom  of  expression  (i.e. 
identity  formation).  While  most  of  the  students  found  learning  within  a virtual  space  compelling,  there  were  the  few 
students  who  found  the  environment  intimidating  and  could  not  learn  to  use  the  technological  functions  found  within 
the  game.  The  initial  technophobia  expressed  by  several  preservice  teachers,  was  finally  alleviated  through  a 
discussion  board  and  partnering  with  classmates.  As  such,  even  in  the  3D  virtual  environment,  players  needed  the 
social  support  for  learning  as  typically  found  in  a face-to-face  environment  (classroom). 

Connection  to  learning.  After  building  a background  for  the  preservice  teachers  by  reading  literature  which 
extolled  the  virtues  of  digital  learning  and  new  literacies,  many  of  the  preservice  teachers  simply  did  not  see  the 
connection  between  playing  in  the  virtual  space  and  fifth-grade  learning.  The  lack  of  connection  to  learning  which 
occurs  in  the  elementary  classroom  may  be  due  to  the  fact  that  people  still  see  games  and  simulations  as  leisure 
activities  and  use  them  as  rewards  for  completing  more  traditional  assignments  such  as  essays  or  reading 
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assignments.  For  most  of  the  preservice  teachers,  vesting  student  time  in  a 3D  environment  simply  did  not  fit  the 
paradigm  for  traditional  learning.  Perhaps  the  failure  to  shift  paradigms  was  due  to  the  fact  that  most  university 
classrooms  still  modeled  effective  instruction  through  the  use  of  traditional  literature  such  as  novel  studies  and  paper 
and  pencil  activities.  Transitioning  to  classroom  settings  which  include  advanced  technology  such  as  simulations 
must  also  come  with  a shift  in  theories  behind  basic  learning.  Teachers  must  first  be  able  to  understand  the  notion  of 
constructivist  learning  as  opposed  to  behaviorist  activities. 

This  reticence  with  the  use  of  virtual  worlds  to  enhance  learning  is  expressed  through  three 
sentient  comments  from  our  participants, 

“Linking  technology  with  literacy  often  heightens  fears  for  teachers  who  are  not  technologically 
educated.” 

“Many  teachers  do  not  want  to  take  that  (Quest  Atlantis)  route  because  they  (teachers)  are  not 
competent  in  their  computers  skills.  For  fear  of  being  so  far  behind  in  the  world  today,  veteran 
teachers  prefer  to  stick  with  the  paper  and  pencil  approach  because  it  has  worked  just  fine  years 
before.” 

“I  am  really  worried  that  these  new  programs  are  going  to  replace  me  (the  teacher)  in  the 
classroom.” 

“I  was  never  able  to  successfully  overcome  the  challenges  I faced  while  playing  QA.” 

Reticence  with  technology  seems  to  be  balanced  with  those  participants,  who  enjoyed  the  diversion  of 
using  QA  as  homework  for  their  university  course.  As  one  of  the  successful  participants  stated, 

“I  believe  happy  successful  students  will  be  a far  more  persuasive  argument  for  integrating 
technology  than  anything  I could  say.” 

Connection  between  practice  and  attitudes  about  QA.  In  an  attempt  to  determine  if  there  was  a 
correlation  between  familiarity  with  QA  and  the  positive  comments  from  participants,  we  sorted  data 
according  to  number  of  logins/attempts  and  positive,  negative,  neutral,  and  no  response.  The  data  revealed 
(Table  2)  that  the  optimum  login  of  10-20  elicited  the  highest  number  of  positive  comments  (15), 


Table  2.  Correlation  Between  Comments  and 
Virtual  Activity  


Log  Ins 

Positive 

Comments 

Negative 

Comments 

Neutral 

Comments 

No 

Response 

<10 

8 

6 

2 

6 

10-20 

15 

7 

2 

6 

20-30 

3 

0 

2 

0 

30-40 

0 

0 

0 

1 

therefore,  during  the  second  year  of  this  project  we  have  added  a minimum  of  15  logins/hours  of  play  for  course 
requirements.  Remembering  that  the  original  intention  of  this  project  was  to  simply  familiarize  teachers  with  the  use 
of  video  games  to  support  literacy  instruction  in  general  education  classrooms,  we  learned  that  the  term  “to  become 
familiar”  had  to  be  quantified  in  order  to  support  pedagogical  understanding  for  preservice  teachers.  In  other  words, 
teachers  must  be  prepared  to  invest  time  to  learn  the  intricacies  of  a particular  game  or  a virtual  environment  before 
one  is  able  to  determine  if  that  particular  media  meets  of  a particular  classroom. 

Play  connected  to  “real"  literacy.  Many  participants  expressed  a direct  connection  between  “playing”  in 
Quest  Atlantis  and  literacy  instruction.  Participants  noted  that  QA  forced  them  to  read  and  understand  written 
instructions  (Figure  3)  as  well  as  post  detailed  statistical  reports  to  support  a choice  they  had  made  within  the  virtual 
environment.  The  embedded  nature  of  literacy  within  the  normal  function  of  the  virtual  experience  seemed  to  create 
conditions  where  students  were  compelled  to  think  about  academic  content  in  a dynamic  environment.  As  one  of  our 
older  female  participants  stated. 


148 


As  each  new  generation  makes  its  way  into  adulthood,  they  must  possess  the  skills  of  the  times.  It 
is  essential  as  educators  to  continually  look  ahead  and  as  best  as  we  can  predict  the  skills  the 
future  generations  will  need.  There  is  little  doubt  the  21st  century  generation  needs  technology 
skills  to  navigate  the  world  in  which  they  live.  As  responsible  and  responsive  educators,  teaching 
with  and  through  technology  will  help  ensure  the  future  success  of  our  students. 

This  study  simply  seeks  to  determine  how  to  shift  the  teaching  paradigm  of  learning  through  the  use  of 
technology  as  opposed  to  teaching  with  technology.  The  critical  component  of  asking  teachers  to  make  this 
important  paradigm  shift  is  that  there  is  a difference  between  teaching  the  simple  use  of  technology  and  understand 
the  potential  technology  holds  for  learning.  Perhaps  embedded  learning  is  the  secret  to  real  instruction  for  younger 
students  when  content  is  naturally  embedded  in  the  virtual  space,  literacy  practice  occurs  as  a leisure  activity. 


Figure  3.  Virtual  Newspaper  in  Quest 
Atlantis 


THE  MIEY  ITATS 


Mayor  Enoch  rides  a pi 
fine  line  with  bike  issue  r ,, 


Like  Bikes? 

Editorial  Piece 
By  Mr.  Grant 

M«yor  Enoch  It  pavirtf 
the  road  with  bad 
Idoat.  Hit  mittion  to 
promote  "Speedy 
Spoket"  over  the 
trusty  "Rollin'  Steady" 
bicycles  Is  dangerous  j 
to  our  citizens! 


Rodin'  Steady  hat  always  been  a reliable  and  safe 
choice  for  bicycle  riders  Mayor  Enoch  it  trying  to 
speed  us  down  the  fatt  lane  of  wrong  choices. 
What  evidence  is  he  using  to  justify  Ns  reasoning? 
Is  he  Involved  with  Speedy  Spoket?  We  mutt  put 
a brake  to  hit  terrible  ideas! 

Mayor  Enoch  it  obviously  recommending  a brand  of  bike  that  it 
inferior  and  unsafe.Why,  at  a community,  are  we  accepting  this? 
Take  a (kick)  stand!  We  should  make  sure  we  are  choosing  the 
safest  bicycle. 


Most  telling  for  this  research  was  the  fact  that  several  of  the  participants,  who  expressed  a sincere  dislike 
for  the  virtual  space  during  the  data  collection  phase  (they  routinely  left  the  computer  lab  in  tears),  later  sought  out 
our  research  team  to  report  on  their  new  found  passion  for  “farming”  (Facebook,  2009)  with  their  friends.  One 
participant  stated,  “I  finally  get  what  you  were  trying  to  tell  us.  You  can  learn  and  relax  at  the  same  time.  This 
(gaming)  is  really  addicting.”  Not  surprising,  for  the  participant  feedback  reflects  what  Turkle  (2005)  found  when 
she  stated, 


There  is  nothing  mindless  about  mastering  a video  game.  The  games  demand  skills  that  are 
complex  and  differentiated.  Some  of  them  begin  to  constitute  a socialization  into  the  computer 
culture;  you  interact  with  a program,  you  learn  how  to  learn  what  it  can  do,  you  get  used  to 
assimilating  large  amount  of  information  about  structure  and  strategy  by  interacting  with  a 
dynamic  screen  display”  (p.  67). 

Perhaps  recognizing  a direct  correlation  between  dynamic  literacy  instruction  and  virtual  spaces  is  as 
difficult  as  recognizing  the  importance  of  literacy  instruction  itself;  one  does  not  comprehend  the  importance  of  a 
skill  until  one  does  not  possess  that  ability.  We  believe  dynamic  play  environments  may  hold  the  key  to  closing  the 
academic  gap  between  groups  of  students  in  American  public  schools.  Virtual  spaces  offer  an  alternative  to 
traditional  static  instruction.  The  shift  to  more  dynamic  learning  environments  simply  has  to  begin  at  the  teacher 
level.  Educators  therefore  need  to  be  able  to  see  how  educational  standards  connect  to  the  learning  which  occurs 
within  virtual  spaces.  Some  of  our  students  are  more  familiar  and  learn  best  through  computer  software  as  opposed 
to  learning  situations  where  printed  text  is  the  sole  source  of  instruction.  We  believe  that  virtual  programs  can 
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enhance  not  replace  traditional  face-to-face  instruction.  As  Smagorinsky  (2009)  states,  educators  must  move  away 
from  thinking  about  reading  and  literacy  instruction  as  “one  dimensional.”  Hull  and  Nelson  ((2005)  note 
There  are  great  challenges  that  accompany  the  incorporation  of  digital  multimodality  into 
classrooms,  challenges  that  are  at  once  technological,  economic,  and  pedagogic.  . . . one  last 
benefit  that  has  to  do  with  multimodality  as  a democratizing  force,  an  opening  up  of  what  counts 
as  valued  communication,  and  a welcoming  of  varied  channels  of  expression  (p.  253). 

There  is  a difference  between  learning  to  turn  on  an  overhead  projector  to  display  a static  image  and 
learning  to  engage  a student  with  a dynamic  media  such  as  a 3D  virtual  environment.  We  contend  that  technology 
instruction  in  schools  of  education  must  move  beyond  familiarity  and  into  deep  understanding  of  the  powerful 
nature  of  well  designed  curriculums  which  incorporate  the  real  potential  of  microcomputer  applications.  The  use  of 
media  and  technology  can  no  longer  be  limited  to  word  processing  and  PowerPoint  presentations.  There  is  a 
substitutive  amount  of  advanced  software  already  available  in  the  commercial  market  already  in  the  hands  of  the 
very  individuals  our  teachers  are  learning  to  instruct. 

Conclusion 

While  some  teachers  welcomed  the  diversion  of  playing  in  a virtual  world,  years  two  and  three  data  still 
needs  to  be  collected  and  analyzed  to  determine  common  threads  of  play  and  variations  on  the  use  of  Avatars  for 
instructional  purposes.  We  believe  that  the  inclusion  of  more  participants  (n=200)  will  either  serve  to  substantiate  or 
more  clearly  define  participation  for  the  preservice  teacher  players.  Additionally,  the  change  in  requirement  to  a 
minimum  log  in  of  15  times  may  shift  the  engagement  of  the  participants. 

Because  virtual  spaces  are  in  a vestigial  stage,  it  is  too  early  to  draw  any  definitive  conclusions  on  the 
effects  of  virtual  learning  on  preservice  teachers.  The  response  from  our  participants  was  mixed.  During  the  second 
year  of  this  project,  the  manner  in  which  the  professor  introduces  (1  hour  introduction  in  the  computer  lab)  and 
prepares  the  preservice  teachers  to  engage  with  QA  has  dramatically  shifted  student  attitudes.  Participants  are  now 
expected  to  read  several  articles  which  build  a more  comprehensive  background  and  pedagogical  foundation  for  use 
of  virtual  spaces.  We  predict  skeptics  will  always  be  present  in  our  sample  of  participants  regardless  of  the  type  of 
foundation  or  schema.  A small  component  of  our  classes  will  always  remain  ambivalent  about  the  increase  use  of 
technology  in  the  classroom  although  electronic  tools  have  impacted  many  other  aspects  of  their  lives.  From 
interviews  and  classroom  discussions,  the  participants  report  the  routine  use  of  technology  in  the  banking  system 
(paying  bills  online),  shopping  on  the  Internet,  supermarket  self-service  checkout  stations,  and  Global  Positioning 
Satellite  (GPS)  systems.  From  professors  on  the  campus,  there  are  widespread  complaints  about  the  extensive  use  of 
texting  during  class.  The  most  prevalent  and  universally  accepted  electronic  tool  for  this  group  is  Facebook,  perhaps 
because  it  is  one  of  the  most  popular  social  networks  available  free  of  charge.  When  asked  what  the  difference  is, 
many  students  pointed  out  how  adding  education  or  obvious  instruction  to  the  equation  takes  away  the  “fun”  from 
any  activity. 

We  are  reminded  again  of  Turkle  (2005),  who  emphasized  the  tenacious  nature  of  attitudes  about  electronic 
and  cyber  tools,  when  she  stated  “parents  want  their  children  to  have  every  advantage,  but  this  new  expertise 
estranges  them.  It  (technology)  seems  to  threaten  a new  kind  of  generation  gap  that  feels  deep  and  difficult  to 
bridge”  (p.  66).  Our  initial  findings  reflect  a similar  “generation  gap”  which  is  somewhat  difficult  to  overcome 
despite  the  course  requirement  of  completing  at  least  5 missions.  While  many  students  liked  playing  Quest  Atlantis 
for  “homework”  many  students  were  still  reticent  to  use  alternative  forms  of  technology  (software)  in  their 
classrooms;  while  at  the  same  time  talking  or  texting  with  a cell  phone.  Virtual  spaces  are  still  seen  as  a form  of 
leisure  and  not  a context  to  support  learning  in  the  classroom.  We  predict,  as  paradigms  on  learning  spaces  shift, 
virtual  learning  will  no  longer  be  a foreign  concept. 


150 


References 


Barab,  S.,  Thomas,  M.,  Dodge,  T.,  Carteaux,  R.  & Tuzun,  H.  (2005).  Making  learning  fun:  Quest  Atlantis,  a game 
without  guns.  Educational  Technology,  Research  and  Development,  53(1),  86-107. 

Berghoff,  B.,  Egawa,  K.,  Harste,  J.,  & Hoonan,  B.  (2000).  Beyond  reading  and  writing:  Inquiry,  curriculum,  and 
multiple  ways  of  knowing.  Urbana,  Illinois:  National  Council  of  Teacher  of  English. 

Bonk,  C.  (2009)  The  world  is  open:  How  web  technology  is  revolutionizing  education.  Jossey  Bass,  San  Francisco, 
CA. 

deCharms  (1968).  Personal  causation.  New  York:  Academic  Press. 

Deci,  E.  L.  (1981).  The  psychology  of  self-determination.  Lexington,  MA:  Heath. 

Fairclough,  N.  (2003).  Semiotic  aspects  of  social  transformation  and  learning.  In  R.  Rogers  (ed.)  An  introduction  to 
Critical  Discourse  Analysis  in  education  (pp.  51-78).  Mahwah,  NJ:Lawrence  Erlbaum. 

Facebook  (2009,  October  5).  Farmville  [Online  application]  Retrieved  from 
http://apps.facebook.com/farmtown/play/ 

Gee,  J.  (2008).  Being  a Lion  and  Being  a Soldier:  Learning  and  Games.  In  J.  Coiro,  M.  Knobel,  C.  Lankshear  & D. 
Leu  (Eds.),  Handbook  of  Research  on  New  Literacies  (pp.  1023-1048).  New  York:  Lawrence  Erlbaum. 

Goldman,  L.  (19  98).  Child’s  play.  Myth,  nimesis  and  make-believe.  New  York:  Palgrave  MacMillan. 

Hull,  G.  & Nelson,  M.  (2005).  Locating  the  semiotic  power  of  multimodality.  Written  Communication,  22,  224-261. 

Lim,  S.,  & Reeves,  B.  (2005).  Being  in  the  game:  Effects  of  avatar  choice  and  point  of  view  on  arousal  responses 
during  play.  Paper  presented  at  the  Information  Systems  Division  of  the  International  Communication 
Association,  November  1,  2005. 

Mead,  G.  (2002).  The  self.  In  C.  Calhoun,  J.  Gerteis,  J.  Moody,  S.  Pfaff,  K.  Schmidt  & I.  Virk  (pp.  240-252) 
Classical  sociological  theory.  Malden,  MA:Blackwell. 

Piaget,  J.  (1962).  Play,  dreams  and  imitation  in  childhood.  New  York:  W.  W.  Norton. 

Rogers,  R.  (2003).  A critical  discourse  analysis  of  literate  identities  across  context:  Alignment  and  conflict.  In  R. 
Rogers  An  introduction  to  Critical  Discourse  Analysis  in  education  (pp.  51-78).  Mahwah,  NLLawrence 
Erlbaum. 

Rotter,  J.  B.  (1966).  Generalized  expectancies  for  internal  versus  external  locus  of  control  of  reinforcement. 
Psychological  Monographs,  80,  1-28. 

Smagorinsky,  P.  (2009).  The  cultural  practice  of  reading  and  the  standardized  assessment  of  reading  instruction: 
When  incommensurate  worlds  collide.  Educational  Researcher,  38(1),  522-527 . 

Spodek,  B.  (1973).  Early  childhood  education.  Englewood  Cliffs,  NJ:  Prentice-Hall. 

Taylor,  S.  E.  (1989).  Positive  illusions:  Creative  self-deception  and  the  healthy  mind.  New  York:  Basic  Books. 

Wallace,  M.  (2006).  Social  studies:  All  day,  every  day  in  the  early  childhood  classroom.  Clifton  Park:  NY: 
Thompson. 

Yellin,  D.,  Jones,  M.  & DeVries,  B.  (2007).  Integrating  the  language  arts  (4th  Ed.)  Scottsdale,  AZ:Holcomb 
Hathaway. 

Zieehezarjeribi,  Y.,  Graves,  I.,  & Gentry,  J.  (in  press).  From  theory  to  practice:  Repurposing  COTS  games  for  preK- 
12.  In  A.  Hirumi  (ed.)  Playing  [video]  games  in  schools:  Engaging  learners  through  interactive 
entertainment,  International  Society  for  Technology  Education  (ISTE). 


151 


Shifting  the  Story  from  Private  to  Public: 
Resistance  to  Narrative  Voice  in  Digital  Storytelling 

Martha  R.  Green,  M.Ed. 

Department  of  Educational  Psychology 
College  of  Education  and  Human  Development 
Texas  A&M  University 

Office  of  International  Outreach 
Annenberg  Presidential  Conference  Center 
1245  TAMU 

College  Station,  TX  77843-1245 
(979)  862-6700 
mgreen@ipomail.tamu.edu 

Shannon  Burnett,  M.A. 

Department  of  Teaching,  Learning  and  Culture 
College  of  Education  and  Human  Development 
Texas  A&M  University 

331  Harrington  Office  Building 
4232  TAMU 

College  Station,  TX  77843-1245 
(979)  862-4978 
sbumett@neo.tamu.edu 


Abstract 

This  qualitative  case-based  study  investigates  digital  storytelling  as  a strategy  to  increase  narrative  voice,  the 
relationship  between  the  narrator  and  the  narrative  act  which  is  captured  when  the  narrator  is  a character  in  the  story; 
and  encourage  reflexivity,  the  ability  to  relate  to  oneself  externally.  The  nature  of  digital  storytelling  shifts  narrative 
from  private  to  public.  Researchers  use  postmodern  philosophy  and  narrative  theory  to  consider  narrative 
representation  as  it  relates  to  self-construction  and  resistance  to  reveal  self;  the  relationship  between  conferencing 
and  disclosure;  resistance  to  narrative  presence;  as  well  as  ethical  issues  related  to  self-revealing  in  the  learning 
environment. 


Introduction 

Digital  storytelling  is  an  expressive  and  culturally  authentic  form  of  representation  that  integrates  personal 
narrative  writing  with  digital  technology.  Pre-service  teachers  develop  planning  skills,  learn  to  understand  the 
relationship  between  words  and  images  to  convey  meaning,  and  come  to  value  digital  technologies  as  tools  for 
planning,  writing,  and  sharing  ideas  as  they  construct  digital  stories.  Digital  storytelling  is  an  adaptation  of  oral 
storytelling  that  utilizes  multimedia  and  telecommunication  tools  to  engage  participants  in  authentic  learning 
experiences  that  provide  real  world  relevance  to  the  learner  within  a situated  context  (Bruner  1996;  Brown,  Collins, 
& Duguid,  1989;  Kearney  & Schuck,  2006;  Emihovich  & Lima,  1995;  Lambert,  2006).  Designing  a digital  story 
requires  participants  to  write  a story  script,  record  a voiceover  and  then  select  images,  video,  and  music  to  elaborate 
the  text.  Unlike  oral  stories  that  are  subject  to  varying  interpretations  and  emphasis,  digital  stories  become 
permanent  artifacts,  a finished  product  that  captures  a specific  moment  in  time  (Lathem,  Reyes,  & Qi,  2006).  The 
digital  artifact  represents  one  telling  of  an  experience.  “A  digital  story  becomes  an  object  that  stands  alone,  making 
it  accessible  for  personal  reflection  and  critique”  (Davis,  2004). 
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Pre-service  teachers  in  training  today  will  in  the  near  future  be  responsible  for  teaching  in  our  public 
schools  where  superior  writing  instruction  in  desperately  needed  (Street,  2003).  Learning  how  to  write  is  a complex 
activity.  The  belief  that  writing  is  a learnable  skill  as  opposed  to  a fixed  ability  is  particularly  important  for  pre- 
service teachers  since  their  own  beliefs  have  the  potential  to  affect  the  learning  of  future  students  (Hammann,  2005). 
“As  national  and  state  standards  require  teachers  to  incorporate  technology  into  teaching,  methods  courses  in 
English  language  arts  as  well  as  all  content  areas  of  teacher  education  must  be  redesigned  to  infuse  technology  into 
all  aspects  of  lesson  design,  instruction,  and  assessment”  (Bowman,  2000,  p.  98),  so  that  pre-service  teachers  come 
to  regard  digital  technology  as  an  effective  tool  for  the  construction  of  knowledge  (Jonassen,  Carr  & Yueh,  1998). 

The  digital  storytelling  process  encourages  the  development  of  narrative  voice  and  reflexivity.  However, 
some  participants  express  resistance  to  the  public  nature  of  digital  storytelling,  either  through  a resistance  to 
narrative  presence,  self-revelation,  or  sharing  with  others.  Conferencing  with  participants  and  peer  feedback 
facilitate  disclosure.  In  situations  when  participants  continue  to  express  resistance,  ethical  issues  must  be  considered. 

Theoretical  framework 

Digital  storytelling  promotes  the  development  of  narrative  voice  (Hofer  and  Swan,  2006).  The  Northwest 
Regional  Education  Lab  (2001)  defines  narrative  voice  as,  .the  writer  coming  through  the  words,  the  sense  that  a 
real  person  is  speaking  to  us  and  cares  about  the  message.  It  is  at  the  heart  and  soul  of  writing,  the  magic,  the  wit, 
the  feeling,  the  life  and  breath.  When  the  writer  is  engaged  personally  with  the  topic,  he/she  imparts  a personal  tone 
and  flavor  to  the  piece  that  is  unmistakably  his/hers  alone.  And  it  is  that  individual  something,  different  from  the 
mark  of  all  other  writers,  that  we  call  voice”  (as  cited  in  Barrett,  2005).  Vygotsky  (1986)  states  that  language  is  the 
primary  way  in  which  human  beings  create  meaning.  Language  mediates  both  communication,  which  enables 
thinking  with  others,  and  also  the  inner  speech  through  which  individual  thinking  is  brought  under  conscious  control 
(Vygotsky,  1986,  Wells,  2007).  The  development  of  higher  order  thinking  in  human  beings  occurs  through  the 
internalization  of  symbolic  tools  such  as  language,  signs  and  images  which  serve  as  abstract  tools  in  changing  the 
nature  of  mental  functioning  (Vygotsky,  1986). 

Bruner  (1991)  contends  that  human  beings  organize  their  experience  and  memory  of  human  happening  in 
the  form  of  narrative  which  can  take  the  form  of  stories,  excuses,  myths  or  reasons  for  doing  things  (Bruner,  1991). 
Stories  serve  as  an  organizing  principle  for  human  action  and  a focus  for  constructing  meaning  from  experience 
(Bruner,  1986,  1996).  Narratives  serve  as  a set  of  interpretive  procedures  for  considering  departures  from  norms 
that  are  meaningful  in  terms  of  established  patterns  of  beliefs.  According  to  Bruner  (1991),  our  stories  are  versions 
of  reality;  they  are  culturally  situated,  and  depend  on  the  human  ability  to  process  knowledge  in  an  interpretive  way. 
Stories  are  how  individuals  recount  their  histories.  What  an  individual  emphasizes  and  leaves  out  of  a life  story  as 
well  as  the  stance  he/she  takes  as  a victim  or  a liberated  individual  establishes  the  teller’s  identity  (Rosenwald  & 
Ochberg,  1992  as  quoted  in  Riessman,  1993). 

The  storytelling  process  also  encourages  reflexivity  (Linde,  1993).  Storytelling  serves  to  reinforce  and 
magnify  a student's  ability  to  reflect  on  experience  and  personal  growth  (Lathem,  Reyes,  and  Qi,  2006).  Reflective 
writing,  or  reflexivity,  includes  self-assessment  and  pushes  the  writer  to  deeper  self-knowledge  (Davis  & Waggert, 
2006). 

The  digital  storytelling  process  shifts  narrative  from  the  personal  to  the  public;  digital  stories  are  created 
with  the  intention  of  sharing  them.  This  is  in  sharp  contrast  to  traditional  writing  activities,  which  most  often  remain 
a private  communique  between  teacher  and  student.  Telling  a personal  story  becomes  a social  process  for  making 
lived  experience  understandable  and  meaningful.  By  making  the  details  of  one’s  life  accessible  to  others  in  public 
discourse,  personal  narratives  bridge  the  dominions  of  public  and  private  life  (Ellis  & Bochner,  1992).  Benmayor 
(2002)  indicates  that  the  digital  medium  allows  students  to  tell  identity  stories  and  theorize  them  at  the  same  time; 
thus  becoming  a gateway  for  constructing  self  and  also  for  contributing  a new  generational  perspective  on  identity, 
community,  belonging  and  selfhood  (Weis,  Benmayor,  O'Leary  & Eynon  (2002). 

Lassonde  (2006)  states  that  writers  use  “positional  writing  practices”  to  frame  their  perception  of  self, 
society,  and  relationship  to  others.  This,  by  nature,  is  a personal  activity.  In  contrast,  the  public  dimension  of  the 
digital  storytelling  medium  enhances  discomfort,  or  even  pain,  for  some  participants  as  they  realize  that  their  work 
has  an  audience  (Gere,  2001).  Bleich’s  (1995)  “pedagogy  of  disclosure”  emphasizes  the  need  for  a community  that 
welcomes  the  diversity  and  individuality  of  storytellers. 

Pre-service  teachers  become  more  willing  to  share  stories  when  conferencing  is  emphasized  during  the 
writing  process  (Tompkins,  2007).  When  a facilitator  conferences  with  a student,  serving  as  a guide  or  mirror  to  aid 
reflection,  students  become  more  comfortable  telling  stories  about  their  own  lived  experiences.  Peer  modeling 
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informs,  motivates  and  provides  information  about  what  actions  lead  to  success  (Pajares,  2003;  Schunk,  2003). 
Group  share  is  a crucial  part  of  the  writing  process  model  (Calkins,  1986;  Tompkins,  2007). 

Methodology 


Participants 

The  researchers  facilitated  a digital  storytelling  project  with  six  groups  of  undergraduate  pre-service 
teachers  in  a reading  education  course  over  three  semesters.  Case  studies  were  selected  from  this  population.  The 
course  is  one-semester  introductory  children’s  literature  course  designed  to  prepare  pre-service  teachers  in  Early 
Childhood  Education  to  teach  critical  reading,  language  arts,  and  children’s  literature.  The  course  is  project-based 
and  emphasizes  both  competent  writing  skills  and  effective  methods  to  teach  writing  to  early  childhood  - 4th  grade 
students.  The  digital  storytelling  project  is  the  culminating  project  of  the  semester.  The  project  draws  on  knowledge 
from  prior  projects:  picture  story  books  and  oral  storytelling  methods,  literary  elements  and  narrative  structure,  and 
the  reading/writing  workshop  process.  The  digital  storytelling  project  shifts  the  application  of  this  prior  content 
knowledge  into  the  digital  writing  environment  and  engages  pre-service  teachers  in  the  development  of  a meaningful 
project  that  integrates  digital  technology  with  narrative  writing  and  demonstrates  how  digital  technology  can  be 
effectively  utilized  in  the  classroom. 

Design  and  procedure 

The  directed  inquiry  task  requires  pre-service  teachers  to  respond  to  a focused  writing  prompt:  Write  a 
200-500  word  personal  narrative  that  represents  a “well-remembered  event”  in  your  life  (Carter,  1993).  This 
personal  narrative  becomes  the  basis  for  a digital  story.  The  narrative  must  be  written  in  the  present  tense  using 
active  voice.  The  personal  narrative  is  recorded  in  the  student’s  own  voice;  then  combined  with  images,  video,  and 
music  to  create  a digital  story.  We  define  a digital  story  as  a short,  reflective,  multimedia  narrative  edited  on  a 
computer,  using  still  images,  voice,  video,  music,  and  sound  to  communicate  with  an  audience  (Lambert,  2005, 

Pauli,  2002). 

Voluntary  interviews  with  study  participants  used  a semi-structured  script.  Specific  areas  of  interest 
included  the  planning  process,  story  content,  and  the  role  of  the  author  in  the  positional  writing  process  (Lassonde, 
2006;  Merriam,  1998).  The  constant-comparative  method  of  data  analysis  was  used  to  code  and  categorize  the 
interview  data.  For  the  purpose  of  this  paper,  pseudonyms  have  been  used  to  protect  the  privacy  of  participants. 
Direct  quotations  and  writing  samples  from  participants  are  presented  in  italics. 

Instructional  strategies 

Virtual  storytelling  circle:  The  story  telling  circle  is  an  integral  part  of  the  digital  storytelling  process  to 
allow  participants  to  share  initial  stories  and  receive  feedback  from  others  (Lambert,  2006).  Due  to  time 
constraints,  the  researchers  utilized  a “virtual  storytelling  circle.”  Students  posted  their  stories  on  the  class  website. 
Using  peer  review  techniques,  each  pre-service  teacher  gave  constructive  feedback  to  two  postings  providing 
suggestions  and  comments  as  a guide  to  the  author. 

Interventionist  conferencing  model:  Researchers  adopted  Atwell's  (1998)  interventionist  approach  to 
conferencing  to  help  students  narrow  story  scripts  and  bring  focus  to  story  themes.  The  interventionist  model 
emphasizes  expert  demonstration,  an  apprenticeship  approach.  Atwell  (1998)  advocates  intervening  directly  in 
students’  writing  by  telling  writers  what  works  and  what  doesn’t. 

Results 

In  facilitating  the  digital  storytelling  process,  the  researchers  recognize  certain  elements  of  resistance  in 
participants'  writing.  Pre-service  teachers  frequently  exhibit  reticence  during  story  selection  and  resist  playing  an 
active  role  in  the  story,  often  because  these  stories  will  be  shared  publicly.  Other  manifestations  of  resistance  are  the 
use  of  past  tense,  passive  voice,  and  third  person  in  writing.  Some  participants  also  resist  the  public  nature  of  digital 
storytelling:  sharing  their  digital  stories. 
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Story  selection 

Social  positioning  in  stories  is  used  by  people  to  distance  themselves  from  situations  they  find  difficult  to 
talk  about  (Harre  & van  Langenhov,  1999).  Rogers  (1992)  suggests  that  what  cannot  be  said  overtly  in  subject 
clauses  often  gets  revealed  more  covertly  and  implicitly  in  object  clauses.  She  explains,  “What  is  unsayable  lies  just 
beneath  the  surface  of  conscious  knowing'’  (Rogers,  as  quoted  in  Josselson  & Hieblich,  1999). 

Nan  is  a 21  year  old  Hispanic  pre-service  teacher.  Nan  was  initially  reluctant  to  tell  a story  about  her  first 
job  as  an  assistant  dance  instructor  at  a summer  camp  for  disabled  children.  I’m  not  very  open  with  people.  I love 

people,  but  people  judge,  and  I feel  like What  are  they  going  to  say  about  my  story?  It  is  just  a little  intimidating 

to  share.  I felt  like  they  were  going  to  think  it  was  cheesy  or  something.  Nan  did  not  post  her  story  on  the  virtual 
circle  to  be  read  by  classmates;  instead  she  e-mailed  it  to  the  teacher/researcher  for  comments.  I e-mailed  Ms. 
Burnett  and  told  her  that  I was  having  trouble.  She  read  the  story  and  said  she  loved  it.  She  said  don ’t  switch.  You 
need  to  focus  on  how  you  were  feeling.  Start  with... as  the  curtain  opened.  Her  telling  me  that  motivated  me  to  not 
switch  again.  “Narrators  select  and  assemble  experiences  and  events  so  they  contribute  collectively  to  the  intended 
point  of  the  story. . .why  is  it  being  told,  in  just  this  way,  in  just  this  setting”  (Mishler,  2000,  pg  8).  How  a narrator 
tells  a story  conveys  a great  deal  about  the  presentation  of  self  (Goffman,  1969).  In  spite  of  Nan's  initial  hesitation 
about  telling  her  digital  story.  Not  Your  Typical  Dance  Recital,  she  reveals  that  the  reflective  writing  process  lead 
her  to  greater  self  understanding.  Sharing  my  experience  as  a dance  teacher  for  disabled  kids  tells  a lot  about  who  I 
am  and  my  personality.  I wrote  about  the  dance  recital  that  I lead  with  Makayla  because  she  taught  me  so  much 
about  the  world,  despite  her  disability.  It  was  through  this  experience  that  I found  out  how  much  I love  working  with 
children.  Nan  comments  on  what  it  meant  to  reflect  on  this  experience,  I have  definitely  grown  as  a person.  I was  a 
role  model  for  these  kids,  especially  for  Makayla.  She  wanted  to  be  exactly  like  me,  and  she  did  everything  I did. 
Maybe  that  part  of  me  because  of  this  experience  has  made  me  grow  into  a role  model  teacher.  It  has  made  me  grow 
and  want  to  work  with  kids.  Her  statement  indicates  how  she  has  learned  from  experience  and  how  her  past 
experience  serves  to  guide  her  actions  in  the  present  (Bloom,  1998,  2000). ’’Man  is  characterized  above  all  by  his 
going  beyond  a situation,  and  by  what  he  succeeds  in  making  of  what  he  has  been  made”  (Satre,  1960,  1963  as 
quoted  in  Bloom,  2000). 

Resistance  to  narrative  presence 

Lana  is  a 20  year  old  Caucasian  pre-service  teacher.  Like  Nan,  she  experienced  difficulty  with  social 
positioning.  The  story  she  posted  on  the  virtual  story  circle  begins. . ..Once  every  year  Mother  Nature  gives  Texas  a 
beautiful  gift  unlike  any  other:  the  gift  of  the  Bluebonnet.  As  if  planted  by  angels  overnight,  these  succulent 
blossoms  appear  everywhere  - on  the  highways,  in  empty  fields,  and  in  backyards.  The  Bluebonnet  description  is 
followed  by  a quote. . .the  voice  of  Lana’s  mother.  “Okay,  Lana  and  Chris,  we're  going  to  take  pictures  in  the 
Bluebonnets  after  church  today.”  Lana  does  not  enter  her  story  until  the  third  paragraph.  I hate  having  my  picture 
taken.  I have  hated  it  from  the  time  when  I was  young.  In  the  final  paragraph,  she  describes  her  mother’s  annual 

photo  ritual.  My  brother  and  were  put  into  position  like  dolls,  while  my  parents  took  pictures What  I really  hated 

was  the  fact  that  I was  being  placed  in  a situation  where  who  I was  as  a person  was  being  pictured  in  one  instant.  I 
am  more  than  a picture,  and  I resent  not  being  seen  as  such.  In  a tone  of  passive  acceptance,  Lana  concludes.  But 

this  is  the  way  the  world  turns I did  learn  to  accept  that  they  (photos)  would  come  with  the  sweetness  of 

springtime  and  the  Bluebonnets.  Peer  feedback,  provided  through  the  virtual  storytelling  circle,  indicates  that  the 
Bluebonnet  description  is  picturesque,  but  the  story  needs  to  be  more  direct  and  concise.  One  student  comments.... 
“Since  your  story  is  actually  about  your  dislike  of  pictures,  your  first  paragraph  feels  like  it  doesn't  really  connect  or 
add  to  the  rest  of  the  story.  In  conferencing  with  Lana,  the  researcher  asks  why  she  is  telling  the  story  and 
encourages  her  to  focus  on  her  dislike  for  picture  taking.  Though  her  final  version  begins  with  the  sound  of  her 
mother’s  voice,  “Lana,  we’re  going  to  be  late,”  Lana  quickly  enters  the  story  and  reveals  her  inner  conflict.  Can  I do 

this?  Today  I will  take  my  senior  pictures I hate  taking  pictures.  Iam  afraid  of  what  people  will  think  of  the 

image,  what  they  will  think  of  me.  I don ’t  want  to  be  seen  as  the  perfect  China  doll.  I just  want  to  be  seen  as  myself. 

Trapped  in  the  moment,  Lana  reflects.  Here  is  my  chance  to  show  the  world....  Transformation  occurs  as 
she  self-assesses  and  asserts  herself.  This  is  my  moment  to  shine,  a moment  that  will  last  forever,  even  when  the 
image  of  youth  falls  far  away.... I am  no  longer  afraid. 

Lana  explains  her  initial  reluctance  to  take  an  active  role  in  the  story.  Telling  this  story  made  me  feel 
vulnerable.  I am  a very  personal  person.  I don ’t  like  to  talk  about  myself  in  personal  ways.  Talking  about  my  dislike 
of  pictures  made  me  feel  almost  lacking  in  a certain  area  of  myself,  and  I found  that  difficult.  She  indicates  that  the 
reflective  writing  process  leads  her  to  reevaluate I have  come  to  realize  that  it  isn  ’t  a “lack”  of  anything.  I found 
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that  by  opening  myself  up  to  others,  I learn  more  about  others,  and  can  see  that  we  all  have  those  stories  we  hold 
within  us.  By  sharing  this  story,  I feel  that  I have  been  able  to  better  connect  with  others  around  me,  laughing  off 
those  moments  that  make  us  feel  uncomfortable.  It  feels  good  to  laugh  these  off  because  they  are  what  make  us  more 
reflective  individuals  and  more  confident  in  ourselves. 

Use  of  third  person 

Hay  and  White  (2005)  explain  that  the  use  of  first  person  in  storytelling  is  “as  an  entry  point  to  the 
explorations  of. . .identity”  (pg.  1).  Katie,  a 25  year  old  Caucasian  pre-service  teacher,  exhibits  a resistance  to  use  of 
first  person  in  her  writing.  She  begins  the  first  draft  of  her  digital  story.  Floppy  Laces',  There  was  once  this  sad  little 
girl.  She  was  in  kindergarten  and  was  super  smart  for  her  age.  She  could  already  write  her  name  and  was  even 
beginning  to  read  and  write  sentences  all  her  own.  There  was  only  one  thing  wrong.  She  couldn  ’t  tie  her  shoe! 
Katie’s  use  of  the  third  person  places  her  outside  the  story  as  an  observer.  In  the  interview,  she  explains  that  writing 
in  first  person  is  “too  personal”  and  “the  most  difficult  aspect”  of  the  digital  storytelling  process  for  her,  I don ’t 
write  about  myself  really.  It’s  not  something  I’m  not  accustomed  to... I’m  not  good  at  expressing  myself... It  was  hard 
for  me  to  put  my  thoughts  like  they  were  my  thoughts.  [Instead],  I put  it  in  third  person;  like  I was  writing  about 
someone  else... It  was  less  emotional.  Conferencing  with  the  researcher/teacher  helped  Katie  to  embrace  the  voice  of 
the  first  person  in  her  writing.  She  commented,  At  the  end,  I did  like  it  better,  but  it  was  just  making  myself  do  it 
[writing  in  first  person].  I really  liked  it  at  the  end  when  I saw  it  all  together. 

Use  of  past  tense 

During  multiple  iterations  of  the  digital  storytelling  process,  the  researchers  observed  that  the  most 
common  exhibition  of  resistance  is  a resistance  to  the  use  of  present  tense  in  personal  narrative;  many  participants 
have  expressed  discomfort  to  telling  the  story  in  the  first  person-present  tense.  Katie  expressed  this  resistance  in 
early  drafts  of  her  digital  story.  Katie  describes  one  last  attempt  to  tie  her  shoe.  With  anger  in  her  voice  and  tightly 
closed  eyes  she  said,  ‘Under,  over,  around  the  tree,  through  the  knothole,  pull  and  see.  ’ When  she  opened  her  eyes 
and  looked  down  she  couldn't  believe  it.  Finally  she  had  done  it!  She  had  tied  her  shoe  all  by  herself.  Delconte 
(2007)  explains  that  “simultaneous  present  tense  narration  [eliminates]  the  time  between  experiencing  and  telling;  it 
also  eliminates  the  (potential)  distinction  between  the  location  of  acting  and  the  location  of  telling”  (pg.  430).  The 
researchers  theorize  that  resistance  to  the  use  of  present  tense  is  intended  to  distance  the  participant  from  their  own 
story. 

Public  nature  of  digital  storytelling 

Padma,  a 20  year  old  Asian  female,  is  an  undergraduate  pre-service  teacher.  Padma’s  digital  story. 

Different,  documents  her  experience  following  an  incident  of  bullying,  when  she  realizes  that  she  is  different  from 
her  peers.  Her  digital  story  concludes.  Today,  I realized  that  I am  different.  My  skin  is  much  darker  and  no  one  has 
hair  as  dark  as  mine,  the  food  I eat  is  different,  and  at  home  I don ’t  speak  English.  How  did  I not  see  this  before? 
How  did  I get  to  third  grade  without  noticing  that  I don  ’tfit  in  ? I thought  they  were  my  friends.  Today,  1 learned 
that  I am  different.  Padma  did  not  have  difficulty  writing  her  story,  but  the  researchers  observed  her  obvious 
discomfort  when  her  digital  story  was  shown  in  class.  Following  that  experience,  Padma  initially  exempted  herself 
from  the  interview  process.  Her  e-mail  said,  I do  not  feel  comfortable  talking  about  my  experience.  My  story  is  very 
personal  and  even  writing  it  was  difficult  for  me.  I had  a very  troubling  time  sitting  in  class  watching  it  with 
everyone.  Please  forgive  me... I hope  you  understand.  Although  Padma  and  other  participants  knew  from  the  onset 
of  the  project  that  sharing  was  an  integral  part  of  digital  storytelling,  it  was  the  public  nature  of  the  process  that 
proved  daunting.  Towards  the  end  of  the  next  semester,  Padma  indicated  to  the  researchers  that  she  was  willing  to 
discuss  her  digital  story.  In  the  interview,  she  stated.  This  story  is  something  that  I kind  of  have  tried  to  shut  out  of 
my  mind  because  it’s  not  something  I like.  The  researcher  asked  Padma,  “What  about  this  media  (digital 
storytelling),  what  about  this  class  made  you  feel  like  that  was  the  story  to  tell?”  She  replied,  I felt  that  my  story  fit 
into  the  class  since  we  had  talked  so  much  about  multicultural  issues  and  diversity.  Commenting  on  the  digital  story, 
the  researcher  tells  Padma,  “To  me  what  is  very  powerful  about  your  story  is  its  credibility  and  honesty  and  your 
vulnerability.  You  could  tell  the  story,  and  it  would  be  powerful,  but  the  fact  that  you  created  a digital  artifact  gives 

you  story  resonance.... It  is  dense,  it’s  rich  with  message so  how  do  you  feel  about  other  people  seeing  your 

digital  story?”  At  first,  I didn  ’t  want  anyone  to  see  it,  it  is  so  personal,  but  I think  that  I have  grown  from  the 
experience.  Padma  reveals  that  she  showed  the  video  to  peers  at  the  school  where  she  is  student  teaching.  I think  a 
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lot  of  people  don ’t  understand  that  stuff  like  this  still  happens.  I hope  teachers  watch  it.  I want  people  to  see  it  and 
really  understand  that  whether  they  physically  see  it  in  the  world  around  them  that  it  does  happen,  and  there  are  so 
many  kids  who  are  hurt  the  way  that  I was.  The  researcher  asked  Padma  what  it  meant  to  her  to  tell  her  story.  I 
wasn  ’t  really  hiding  from  it  (being  different ) any  more.  I was  telling  people  about  who  I am  and  where  I come  from. 
I was  opening  up  a side  of  me  that  most  of  them  haven’t  seen.  Explaining  her  discomfort  in  class  when  the  video 
was  shown,  Padma  acknowledges  that  her  story  is  important  for  others  to  hear,  but  avers  that  she  does  not  enjoy 
discussing  her  personal  differences  or  desire  to  be  a focus  of  attention.  I don  ’1  always  like  being  the  one  that’s 
different.  I like  being  able  to  tell  them  my  experiences  so  that  children,  their  students,  will  benefit  from  it,  but  at  the 
same  time,  I’m  not  very  keen  to  hear  what  they  (other  pre-service  teachers)  are  going  to  say  after  that. 

Educational  Importance  of  Study 

Hofer  & Swan  (2006)  indicate  that  iterative  formative  evaluations  help  to  improve  the  implementation  of 
an  instructional  approach  such  as  digital  storytelling  and  lead  to  the  development  of  a theory  of  teaching  and 
learning.  The  educational  importance  of  this  study  resides  in  its  attempt  to  analyze  multiple  implementations  of 
digital  storytelling  and  barriers  to  successful  writing.  The  most  significant  findings  of  the  research  is  the  perceived 
benefit  of  digital  storytelling  as  a pedagogy  to  engage  students  in  a writing  process  that  leads  to  the  production  of  a 
meaningful  artifact  that  can  be  reflected  on  by  the  author  and  shared  with  others  and  to  increase  their  valuing  of 
digital  technology  as  method  to  teach  writing  in  the  classroom. 

Two  areas  of  interest  stand  out  in  the  interview  data.  Students  indicated  that  the  virtual  storytelling  circle 
supported  learning.  Not  only  did  students  read  two  stories  to  provide  feedback;  most  said  they  read  most  all  the 
stories  on  the  class  site.  Reading  other  student’s  stories  helped  students  make  decisions  about  their  own  work.  The 
virtual  story  telling  circle  served  as  an  active  learning  environment.  Students  reported  that  feedback  from  peer 
mentors  was  useful,  but  the  greatest  benefit  came  from  being  able  to  read  many  stories. 

Students  also  talked  about  how  helpful  conferencing  with  the  researchers  was.  Adoption  of  the  Atwell’s 
(1998)  interventionist  approach  to  conferencing  helped  students  to  refine  story  scripts.  Interventionist  conferencing 
took  place  in  the  classroom;  however,  much  additional  conferencing  took  place  in  e-mail  messages  and  postings  to 
the  virtual  story  circle.  Students  indicated  that  receiving  direct  input  about  how  their  writing  could  be  improved  was 
extremely  helpful.  The  positive  relationship  between  the  researchers  and  students,  collaborating  together,  seemed  to 
increase  students’  confidence  in  their  own  writing  abilities. 

Researchers  also  became  more  aware  of  the  relationship  between  the  guiding  role  of  the  facilitator,  who 
serves  as  a mirror  to  aid  reflection  through  conferencing  and  students’  comfort  level  in  telling  stories  about  their 
own  lived  experiences.  Teachers  and  professors  that  decide  to  integrate  digital  storytelling  into  their  classrooms 
must  consider  the  ethical  implications  of  the  process  (Haney,  2004).  This  is  particularly  important  when  students  are 
attempting  to  discuss  emotional  or  distressing  accounts  (Gere,  2001).  Students  resistant  or  pained,  after  repeated 
conferences,  should  be  encouraged  to  consider  other  topics  for  their  digital  stories  (Haney,  2004). 

Writing  visually  produces  a new  form  of  social  text  (Ulmer,  1989,  p 3).  “The  'truths'  of  narrative  accounts 
lie  not  in  their  faithful  representation  of  a past  world,  but  in  the  shifting  connection  they  forge  between  past,  present, 
and  future"(Riessman,  2001,  p 342).  In  interviews,  participants  revealed  that  they  felt  empowered  by  the  process  of 
constructing  a digital  story  and  sharing  the  videos  with  peers.  Pre-service  teachers  indicated  that  that  they  would  use 
digital  storytelling  in  the  future  with  students.  Most  importantly,  they  came  to  understand  that  the  stories  we  tell 
allow  us  “to  know  ourselves  and  others  and  the  positions  from  which  we  speak”  (Ellis  & Flaherty,  1992). 
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Abstract 

The  purposes  of  the  project  were:  (1)  to  create  a social  networking  forum  that  would  promote  the 
Instructional  Design  and  Technology  Department  (IDT)  undergraduate  program  to  high  school  students,  community 
college  students,  transfer  students,  and  the  parents  of  prospective  students,  and  (2)  to  provide  graduate  students  an 
opportunity  to  work  with  “a  real”  client  (a  faculty  member)  on  an  actual  project  management  scenario  that  involved 
the  use  of  instructional  project  management  principles  and  skills.  During  their  project  management  course,  two 
graduate  students  used  a systematic  approach  to  this  project  (ADDIE/IPECC)  starting  with  the  assessment  of  the 
departments’  need  and  prospective  students’  needs  and  ending  with  presenting  the  product  to  the  client  and 
documenting  it.  The  Facebook  page  provided  information  about  the  program  including  links  to  enrollment,  campus 
life  and  environment,  and  multimedia  examples  of  past  students’  work.  This  project  provided  the  graduate  students 
an  opportunity  to  discuss  general  issues  in  instructional  project  management  as  well  as  to  practice  mentoring 
behavior  with  the  graduate  student  peers  in  order  to  gain  experience  in  management  roles.  Findings  of  the  Facebook 
project  and  implications  for  increased  interactive  recruitment  through  social  networking  for  higher  education 
institutions  will  be  detailed. 


Project  background 

Graduate  students  enrolled  in  a project  management  course  to  experience  how  to  implement  a project 
management  activity  in  a real  world.  The  course  that  allowed  the  students  to  go  through  a project  management 
simulation  is  critical  in  order  to  give  the  students  knowledge  and  skills  needed  by  real  fields  of  work.  The 
Instructional  Design  and  Technology  Department  (IDT)  provided  the  students  an  opportunity  to  work  with  the 
department  to  promote  the  department  undergraduate  program  online  through  a social  networking  tool.  Working 
with  a real  client  in  this  case  an  IDT  faculty  member  supported  students’  activities  and  outcomes. 
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Prior  to  the  project,  the  IDT  Department  had  promoted  their  department  the  university’s  website  and 
through  printed  materials.  The  department  chair  and  faculty  member  wanted  to  promote  the  undergraduate  program 
through  social  networking  means  as  well. 

Today,  there  are  many  sites  on  the  internet  that  are  visited  by  many  people  globally  to  get  information  and 
to  connect  with  others.  One  of  the  most  popular  sites  currently  used  by  both  younger  and  older  generations  is 
Facebook.com.  This  trendy  networking  tool  provides  various  options  for  connection  with  old  and  new  friends,  self 
promotion,  distribution  of  information  to  the  masses,  and  even  specific  forums  creation.  The  use  of  social 
networking  tools  is  increasing  among  students,  with  a whopping  85%  of  college  students  using  Facebook  according 
to  Social  Media  University’s  Lee  Aase  (Ward  2008).  In  2008,  the  Next-Student  Student  Loan  Blog  described  an 
increasing  number  of  higher  education  admission  offices  who  are  using  Facebook  to  recruit  and  interact  with 
prospective  students.  Social  networking  sites  are  where  many  incoming  students  are  going  to  for  social  interaction, 
information,  and  informal  learning  (“Networking  Sites. ”-2007).  Based  on  the  popularity  of  Facebook,  the  IDT 
Department  proposed  the  promotion  of  the  IDT  program  through  a Facebook  group  and  link  it  to  create  links  to  the 
department’s  website. 


Project  goals  and  objectives 

The  main  goal  of  this  project  was  to  create  a social  networking  forum  that  would  promote  the  IDT 
departments’  undergraduate  program  to  high  school  students,  community  college  students,  transfer  students,  and 
parents  of  prospective  students.  The  other  goal  of  this  project  was  to  provide  graduate  students  an  opportunity  to 
work  with  a “real”  client  (a  faculty  member)  on  an  actual  project  management  scenario  that  involves  the  use  of 
instructional  project  management  principles  and  skills. 

Specifically,  this  project  has  the  following  objectives: 

1 . Organizational  objectives:  a)  increase  communication  between  the  department  and  students,  and  b) 
promote  the  IDT  Department  widely  through  Facebook;  and 

2.  Instructional  objectives:  a)  apply  the  project  management  theories  into  real  word  situations,  b)  gain 
experience  in  working  with  a real  client,  and  c)  turn  client’s  needs  into  project  outcomes. 

Project  management  and  project  development 

Wysocky  (2007 ) defined  project  management  as  “a  method  and  set  of  techniques  based  on  the  accepted 
principles  of  management  used  for  planning,  estimating,  and  controlling  work  activities  to  reach  a desired  end  result 
on  time-within  budget  and  according  to  specification”  (p.  34).  In  order  to  plan,  estimate,  and  control  the  project 
activities,  a method  and  techniques  based  on  principles  of  management  are  needed  to  reach  the  project  goals.  To 
develop  a project,  one  must  not  only  apply  the  project  development  activities  but  also  apply  the  project  management 
activities  as  a framework  of  project  development  activities. 

According  to  Russell  (2002),  project  management  consists  of  activities  to  manage  a project,  including 
planning,  organizing,  and  controlling  work.  Project  development  consists  of  activities  to  develop  a project  and  it 
involves  the  analysis  of  skills  or  a knowledge  gap,  design  of  the  solution,  building  and  testing  the  solution.  Lynch 
and  Roecker  (2007)  stated  that  good  instructional  design  and  good  project  management  are  different.  They  are  not 
interchangeable  although  they  may  go  hand  in  hand. 

Lynch  and  Roecker  (2007)  described  the  relationship  between  the  project  development  procedure  called  the 
ADDIE  model  (Analysis,  Design,  Develop,  Implement,  and  Evaluate)  and  the  project  management  model  called  the 
IPECC  model  (Initiating,  Planning,  Executing,  Closing,  Controlling)  in  managing  and  developing  a project.  The 
relationship  between  the  two  models  can  be  seen  in  each  phase  of  ADDIE  and  IPECC  models.  In  the  first  phase  or 
initiating  phase  of  IPECC  model,  the  project  goal  is  established  and  the  project  objectives  are  determined.  The 
second  phase  or  planning  phase  of  IPECC  model  is  related  to  the  analysis  stage  of  the  ADDIE  model.  The  third  and 
the  fourth  phase  of  IPECC  model  which  are  executing  and  controlling  phase  are  connected  to  the  design  and  develop 
stages  of  the  ADDIE  model.  The  fifth  phase  or  closing  phase  of  IPECC  model  is  related  to  the  implementation  stage 
of  the  ADDIE  model.  The  last  stage  of  the  ADDIE  model  that  is  the  evaluation  consists  of  the  project  evaluation 
report  and  project  documentation  activities. 
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Method 


For  this  project,  the  graduate  students  used  a systematic  approach  that  is  integrated  ADDIE-IPECC 
approach.  They  used  a combination  of  the  ADDIE  model  ( Analysis,  Design,  Develop,  Implement,  and  Evaluate)  and 
the  IPECC  model  (Initiating,  Planning,  Executing,  Controlling,  and  Closing). 

Initiating 

In  the  initiating  phase  of  the  ADDIE-IPECC  model,  the  graduate  students  conducted  five  tasks,  involving 
stating  the  opportunity,  establishing  the  project  goal,  determining  the  project  objectives,  establishing  the  client’s 
need,  and  creating  a request  for  proposal  (RFP)  that  shows  a clear  statement  of  need  and  to  have  external  input  for 
the  project. 

Planning  (analysis) 

In  the  planning  phase,  another  student  team  reviewed  the  RFP  for  the  Facebook  page,  identified  the  project 
activities,  estimated  the  activities  duration  (timeline)  (as  seen  in  Figure  1),  determined  resource  requirements 
(analysis  the  Facebook  page  and  the  materials  needed  for  the  page),  created  communication  plan,  created  a risk 
management  plan,  created  budget  worksheet,  created  the  criteria  for  level  of  acceptance,  created  a proposal,  and 
presented  the  proposal  to  the  client.  After  the  proposal  was  approved  by  the  client,  the  students  who  created  the  RFP 
also  created  the  contract  and  gave  it  to  the  client  to  be  reviewed  and  signed. 

Executing  and  controlling  (design  and  development) 

In  the  executing  phase,  the  graduate  students  used  the  other  team’s  proposal  to  design  and  develop  the 
Facebook  page.  In  addition,  the  graduate  students  modeled  the  department’s  Facebook  after  the  university  existing 
Facebook  page.  The  University’s  Facebook  page  had  been  named  the“College  and  University:  Facebook  Page  of  the 
Month”  for  January  2009  (“College  Admission  Offices  Using  Facebook,  2008;  “Universities  of  the  Month,”  2009). 
The  university’s  Facebook  page  can  be  seen  in  Figure  2.  Facebook  commended  the  University  for  its  Facebook  page 
organization  and  “personality”  of  the  page.  The  AECT  WIU  student  chapter  also  already  had  presence  which 
featured  the  events  of  the  students’  organization  and  information  about  the  national  AECT  organization  as  seen  in 
Figure  3. 
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Figure  1.  Project  timeline. 
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For  the  IDT  Department  undergraduate  program  Facebook  page  design,  we  chose  the  Facebook  layout 
(Facebook  Page)  and  created  a storyboard  as  a guideline  to  develop  the  page.  Based  on  the  storyboard,  the  prototype 
was  produced  and  materials  were  collected.  Controlling  was  done  during  the  design  and  development  processes  and 
we  conducted  three  tasks:  establishing  the  progress  reporting  system,  monitoring  project  progress  versus  lesson  plan, 
and  revising  the  project  plan  when  necessary. 

Closing  (implementation  and  evaluation) 

After  we  created  the  prototype  and  collected  the  materials,  we  incorporated  the  materials  on  the  Facebook 
page  and  we  also  created  an  account  for  the  client  on  Facebook  as  seen  in  Figures  4 and  5. 

The  last  phase  of  integrated  ADDIE-IPECC  model  is  evaluation.  In  this  phase,  we  conducted  the  following 
tasks:  evaluating  the  product,  reporting  the  product  evaluation,  signing  the  acceptance  product  document, 
completing  project  documentation,  and  celebrating  the  ending  of  the  project. 

Findings 

Several  findings  emerged  from  this  project.  First,  the  Facebook  page  for  the  IDT  Department  was  created 
and  met  with  the  client’s  needs  and  expectations.  The  client  accepted  the  product  and  signed  the  acceptance  product 
document.  The  client  was  optimistic  that  Facebook  page  would  be  beneficial  for  the  IDT  undergraduate  program  to 
invite  and  attract  prospective  students’  to  consider  and  finally  enroll  in  the  IDT  undergraduate  program.  The 
Facebook  page  was  created  by  considering  its  usability  (easy  to  use,  easy  to  learn,  easy  to  navigate),  its  interactivity 
(two-way  feedback),  and  dissemination  of  information. 

The  combination  of  the  two  procedures  helped  the  graduate  student  authors  to  obtain  an  understanding  of 
project  management  and  its  associated  activities.  As  a project  manager  in  instructional  technology  projects,  the 
graduate  students  had  gained  the  hands-on  experience  of  developing  an  entire  project  using  project  management 
skills.  The  responsibility  to  the  project  management  plans,  control  of  the  project  progression,  and  reports/evaluation 
of  the  project  were  main  activities  that  the  students  went  through  to  develop  their  managerial  skills.  In  addition,  they 
also  learned  to  work  cooperatively  and  take  responsibility  as  a team.  This  form  of  learning  allowed  them  to  not  only 
work  together  but  also  to  develop  their  social  and  collaborative  abilities. 


Third,  using  the  integrated  ADDIE  and  IPECC  model,  the  students  learned  how  the  project  management 
activities  and  the  project  development  activities  work  at  the  same  time  in  a-real  world  application.  The  students 
experienced  how  complex  it  was  to  develop  and  manage  a project.  Furthermore,  the  opportunity  to  work  with  a real 
client  trained  their  skills  of  managing  and  developing  a real  project,  including  improving  responsibilities, 
maintaining  an  open  communication  with  the  client,  and  being  flexible  on  the  tasks,  while  staying  focused  on  the 
goals  and  objectives. 
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Conclusion 

Based  on  the  findings,  the  following  conclusions  were  made: 

1.  The  IDT  Department  now  has  social  networking  tool  to  promote  its  undergraduate  program.  Using 
Facebook  as  an  interactive  recruitment  tool  for  an  education  institution  would  be  beneficial  for  the 
institution  itself  and  the  prospective  students  in  term  of  usability,  its  interactivity,  and  dissemination  of 
information. 

2.  Developing  a project  for  a real  client  involved  two  major  activities,  project  management  activities  and 
project  development  activities.  The  integration  of  IPECC  and  ADDIE  models  to  develop  a Facebook 
project  combined  two  systematic  procedures  that  allowed  the  students  to  work  cooperatively  on  a team 
to  meet  the  client’s  expectations. 

3.  The  project  management  course  is  the  framework  and  basis  of  the  project  development  activities.  It  is 
suggested  that  this  course  be  a core  course  for  the  graduate  students  in  the  IDT  Department. 

Practical  Implications 

The  following  are  some  lessons  learned  throughout  the  project  that  would  benefit  the  students  to  manage 
and  develop  future  projects  of  instructional  technology” 

1.  Developing  a social  networking  project  to  promote  organizations  or  institutions  in  a real-world  context 
can  be  done  by  integrating  the  two  procedures,  the  project  management  procedures  (IPECC)  and  the 
project  development  procedures  (ADDIE). 

2.  Applying  a systematic  procedure  to  manage  and  develop  a project  is  crucial 

3.  It  is  essential  to  have  an  opportunity  to  work  with  a real  client  to  train  the  senses  of  managing  and 
developing  a real-world  project  that  would  prepare  the  students  to  be  more  knowledgeable  and  ready  to 
enter  a real  field  of  work. 

4.  In  terms  of  working  with  a real  client,  clearly  stating  the  client’s  needs  and  expectations,  being 
flexible,  and  creating  open  communication,  are  initial  methods  and  strategies  to  direct  the  project  on 
the  right  track. 

5.  In  order  to  give  the  students  a comprehensive  and  complete  understanding  of  the  entire  process  in 
project  management  of  instructional  technology,  the  project  management  of  Instructional  Technology 
course  should  provides  all  graduate  students  with  hands-on  experiences  in  project  management  and 
development  activities. 
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Designing  for  Affect:  The  Potential  of  Social  Networking 
for  Teacher  Education 


Ellen  S.  Hoffman 
University  of  Hawaii  at  Manoa 

Emerging  Web  2.0  tools  are  increasing  the  capability  of  designing  online  learning  that  provides  rich  affective 
environments  that  positively  impact  learning  communities.  Many  advocates  promote  the  use  of  social  networking 
for  community  building,  but  some  critics  have  suggested  that  the  links  between  computer-mediated  communication 
(CMC)  and  deep  learning  or  engagement  are  not  well  documented,  proposing  that  such  advocacy  is  more  a potential 
than  reality  (Godwin,  Thorpe,  & Richardson,  2008;  Rourke  & Kanuka,  2009;  San  Millan  Maurino,  2007). 

In  this  mixed  methods  case  study,  the  author  examined  the  impact  of  implementing  a social  networking  tool 
in  an  online  course  to  determine  whether  there  was  evidence  to  support  positive  impact  on  student  outcomes, 
particularly  in  relation  to  the  affective  domain  of  learning.  Does  social  networking  improve  student’s  sense  of 
belonging  to  a community?  Does  community  building  impact  student  evaluation  of  the  course?  Does  social 
networking  increase  engagement  and  motivation?  Most  important,  does  such  interaction  through  social  networking 
have  an  impact  on  student  learning? 

To  examine  these  questions,  a social  networking  tool,  Ning  (http://www.ning.com),  was  implemented  in  an 
asynchronous  distance  education  college  course  to  supplement  the  tools  in  the  institution’s  online  course 
management  system.  Ning  has  become  a hot  tool  in  educational  settings  over  the  past  two  years.  The  site  ranking 
service,  alexa.com,  shows  Ning  as  the  63rd  most  popular  Web  site  in  the  U.S.  in  the  second  quarter  of  2009.  The 
social  networking  site  has  almost  7,000  Ning  networks  tagged  with  “education”  via  a search  in  September  2009.  Its 
“Classroom  2.0”  site  boasts  30,500  members  and  7,000  members  in  “Ning  in  Education,”  suggesting  the  reach  of 
this  tool  in  both  K-12  and  higher  education. 

The  following  is  a report  on  the  impact  of  using  Ning  in  a summer  online  course  within  the  context  of 
learning  theory  and  technology  affordances  in  course  design.  The  case  study  included  5 1 students  enrolled  in  four 
sections  of  an  undergraduate  summer  2008  class,  “Computers  in  Education.”  The  online  course  was  fully 
asynchronous  with  bi-weekly  modules.  Students  used  multiple  online  collaborative  and  local  productivity  tools, 
although  the  main  content  for  the  educational  technology  course  is  structured  within  a traditional  course 
management  system  (CMS)  supplemented  by  external  tools  including  Ning.  The  potential  for  enhancing  learning 
through  the  intentional  addition  of  tools  to  facilitate  the  affective  dimension  of  learning  is  demonstrated  in  this  study 
of  a teacher  education  course  with  learners  indicating  exuberance  for  the  new  tool,  increased  satisfaction  with  the 
online  course,  and  self-reported  positive  impacts  on  their  learning  resulting  from  a supportive  collaborative 
environment. 


Background 

While  more  students  are  experiencing  distance  education,  few  report  that  they  make  this  choice  primarily 
because  they  like  distance  learning  (DL)  classes.  Typical  reasons  are  more  commonly  related  to  flexibility  in 
scheduling  or  geographic  issues  that  create  barriers  for  reaching  traditional  campus-based  sessions  (Rovai,  Ponton, 
Wighting,  & Baker,  2007).  Even  when  students  indicate  that  they  have  had  a positive  DL  experience,  many  raise  the 
issue  that  it  just  doesn’t  seem  the  same  as  actually  being  there  and  interacting  in  a face-to-face  setting  (Perez -Prad  & 
Thirunarayanan,  2002).  In  a recent  international  review,  Armstrong  and  Franklin  (2008)  indicated  that  even  digital- 
age  students  who  enter  higher  education  are  deeply  enculturated  in  the  traditional  processes  of  schools  and 
classroom  learning,  setting  specific  mental  models  for  expectations  and  interactions.  This  view  includes  the  hidden 
assumption  of  social  interaction  and  engagement  well  understood  within  the  traditional  classroom  that  establishes  a 
framework  for  affective  aspects  of  learning.  Considering  such  factors  in  the  design  of  online  learning  is  proposed  as 
a means  to  increase  social  interaction  and  engagement  (Perez-Prad  & Thirunarayanan,  2002). 

Community  building  in  online  courses  is  a common  recommendation  as  one  way  to  both  improve  learning 
from  a constructivist  perspective  as  well  as  establish  a social  context  in  distance  learning.  Further,  student-to-student 
interaction  has  been  promoted  as  one  of  the  keystones  for  positive  evaluations  of  distance  learning  as  shown  through 
multiple  meta-analyses  of  DL  (Tallent-Runnels,  et  al.,  2006;  Young,  2007;  Zhao,  Lei,  Yan,  Lai,  & Tan,  2005).  In 
addition,  student  satisfaction  was  shown  to  be  related  to  a sense  of  presence,  having  interpersonal  communications 
needs  met,  and  being  treated  as  individuals  (Dennen,  Darabi,  & Smith,  2007;  Mazer,  Murphy,  & Simonds,  2007). 
Today  many  online  classrooms  incorporate  such  common  tools  as  email,  discussion  boards,  and  more  recently. 
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blogs,  wikis  and  various  Web  2.0  tools  that  not  only  can  support  content  learning  but  enhance  interaction. 

Commonly  such  tools  are  used  as  part  of  required  course  discussions,  with  previous  studies  indicating  use  was  more 
prevalent  when  required  and  graded  (Dennen,  2005;  Hamann,  Pollock,  & Wilson,  2006).  Informal  use  was  uneven 
when  the  opportunity  was  made  available  but  not  mandatory. 

Social  network  sites  (SNS)  in  higher  education 

With  the  growth  and  popularity  of  social  networking  external  to  formal  educational  settings,  advocates 
have  argued  that  these  tools  not  only  provide  powerful  affordances  for  community  building  in  distance  learning,  but 
potentially  are  transformational  technologies  for  higher  education  more  generally  (Hart,  2008;  Mason  & Rennie, 
2008).  Social  networking  covers  a wide  range  of  online  environments,  with  many  formal  definitions  broad  enough 
to  encompass  almost  any  Web  2.0  tool  (Alexander,  2006).  Boyd  and  Ellison  (2007)  include  three  criteria  in  their 
definition  of  social  network  sites  (SNSs)  which  are: 

web-based  services  that  allow  individuals  to  (1)  construct  a public  or  semi -public  profile  within  a bounded 

system,  (2)  articulate  a list  of  other  users  with  whom  they  share  a connection,  and  (3)  view  and  traverse 

their  list  of  connections  and  those  made  by  others  within  the  system,  (paragraph  4) 

They  note  that  many  SNSs  allow  users  to  write  comments  on  “friend’s”  profiles  and  send  private  messages 
although  these  are  not  universal  features.  Rather  than  communities  organized  by  topic,  SNSs  are  “structured  as 
personal  (or  "egocentric")  networks,  with  the  individual  at  the  center  of  their  own  community”  (Boyd  & Ellison, 
2007).  Perhaps  more  critical  from  an  educational  viewpoint,  many  of  the  SNSs  are  enhanced  with  multiple 
collaborative  tools  that  go  beyond  the  personal  profile  and  “friending”  links,  including  the  ability  to  post  and  share 
files  (text,  images,  audio  and  video),  participate  in  discussions  or  blogs,  co-create  and  edit  content  with  wiki-like 
tools,  and  link  in  and  tag  external  resources  from  other  web  sites  paralleling  social  bookmarking.  Sites  such  as  Flikr 
or  YouTube,  despite  meeting  the  Boyd  and  Ellison  criteria,  are  in  fact  more  commonly  seen  as  environments 
primarily  for  sharing  content,  digital  pictures  and  video  respectively,  rather  than  SNSs.  Studies  of  users  of  SNSs 
have  shown  SNS  sites  have  the  capacity  to  increase  social  ties  and  interaction,  provide  an  outlet  for  self-expression, 
and  assist  in  helping  with  information  seeking  and  task  completion  (Gallant,  Boone,  & Heap,  2007). 

While  various  public  collaborative  environments  existed  on  the  Internet  as  early  as  the  1980s,  the 
emergence  of  social  networking  as  it  is  best  understood  today  arose  with  the  massive  commercially-supported  sites 
early  in  this  decade,  including  MySpace  in  2003  and  Facebook  a year  later.  By  mid-2009,  these  two  sites  had  rapidly 
moved  to  the  top  five  for  Internet  traffic  in  the  United  States,  and  within  the  top  1 1 internationally  as  reported  by 
alexa.com.  Because  Ning  itself  is  a new  SNS,  limited  peer-reviewed  studies  of  its  use  in  instruction  are  in  print. 
Other  studies  of  teaching  use  of  social  networks  have  been  published  (Mazer,  et  al.,  2007;  Minocha,  2009),  while 
Ning  education  sites  such  as  “Ning  in  Education,”  http://education.ning.com/,  abound  with  personal  testimonies  to 
the  effectiveness  of  Ning  in  teaching. 

Social  networking  provides  novel  affordances  for  computer-supported  collaborative  learning  (CSCL), 
particularly  in  asynchronous  environments  (Grant,  2008;  Idris  & Wang,  2009).  In  a recent  report  studying  the  uses 
of  social  software  in  the  higher  education,  Minocha  (2009)  found  that  educational  goals  for  employing  SNSs 
included  initiating  new  ways  of  learning,  giving  control  to  students,  providing  transferable  skills,  peer-to-peer 
learning,  enhancing  reflective  learning,  creating  a digital  identity,  and  fostering  social  engagement.  The  case  studies 
reviewed  showed  multiple  benefits  in  using  SNSs,  including  retention,  socialization,  collaborative  learning, 
engaging  students,  sense  of  control  and  ownership,  problem-solving  and  sense  of  achievement,  visibility  of  artifacts 
created,  integration  of  multimedia,  adding  novelty  and  excitement  to  the  learning  environment,  overcoming  isolation 
and  geographic  differences,  and  students’  positive  perceptions  of  the  educator  involved  in  SNS  initiatives. 

From  a design  perspective,  these  tools  are  well  suited  to  provide  a learner-centered  orientation  and  support 
both  formal  and  informal  learning  interactions  seen  as  critical  to  community  and  collaborative  meaning-making  in 
constructivist  learning.  Further,  the  public  and  no-cost  nature  of  these  tools  allows  designers  and  instructors  to 
incorporate  them  in  post-secondary  education  without  the  restraints  often  imposed  institutionally  on  what  software 
or  support  resources  are  available  for  instruction,  allowing  freedom  to  mix  and  match  to  meet  specific  course 
objectives  and  pedagogical  strategies  (Boyd  & Ellison,  2007;  Grant,  2008).  Others  have  advocated  for  the  use  of 
social  networks  in  formal  educational  settings  based  on  the  prevalence  of  use  by  the  current  digital  generation  of 
students  and  their  expectations  for  technology  integration  (Hart,  2008;  Pence,  2007).  As  noted  in  a recent  reports  on 
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the  use  of  Web  2.0  tools  in  higher  education,  institutional  supports  are  uncommon  and  most  adoption  of  SNSs  for 
teaching  has  been  at  the  individual  staff  level  (Ajjan  & Hartshorne,  2008;  Armstrong  & Franklin,  2008),  although 
administrative  uses  for  recruiting  and  public  relations  have  grown  over  the  past  year. 

Affective  domains  of  learning  and  SNSs 

The  implementation  of  intentional  affective  strategies  in  the  design  of  online  learning  environments  have 
often  been  ignored  in  higher  education  because  of  a lack  of  good  tools  and  because  of  attitudes  that  equate  adult 
learning  solely  with  cognitive  functioning  (Craig,  Graesser,  Sullins,  & Gholson,  2004;  Main,  1992;  Pierre  & 
Oughton,  2007;  Shephard,  2008).  Pierre  and  Oughton  (2007)  note  the  affective  domain  “refers  to  our  attitudes  and 
willingness  to  take  part  in  new  things,  and  ability  to  make  decisions  about  how  we  operate  and  behave  in  a variety  of 
circumstances”  (p.  3),  so  that  it  is  essential  for  both  motivation  and  cognition.  Hudlicka  (2003)  stated  in  a review  of 
the  psychological  research  on  affect  that  it  is  “critical  for  the  successful  completion  of  a task,  for  avoiding  (often 
disastrous)  errors,  for  achieving  optimal  performance”  (p.  6).  Individuals  in  a positive-affect  condition  reduce  time 
to  task  completion  and  are  more  successful  in  completing  tasks  (Isen  & Reeve,  2005). 

Affective  issues  are  known  to  “exert  powerful  influences  on  learners'  ability  to  engage  with  learning  and  to 
progress”  (Rose  & Meyer,  2002,  Ch.  2).  Rose  and  Meyer  (2002)  argue  that 

understanding  affective  issues  can  help  teachers  support  all  learners  more  appropriately.  Of  the  three 
learning  networks,  affective  networks  are  perhaps  intuitively  the  most  essential  for  learning,  yet  they  are 
given  the  least  formal  emphasis  in  the  curriculum.  All  teachers  know  how  important  it  is  to  engage  students 
in  the  learning  process,  to  help  them  to  love  learning,  to  enjoy  challenges,  to  connect  with  subject  matter, 
and  to  persist  when  things  get  tough.  When  students  withdraw  their  effort  and  engagement,  it  is  tempting  to 
consider  this  a problem  outside  the  core  enterprise  of  teaching.  We  believe  this  is  a mistake.  Attending  to 
affective  issues  when  considering  students'  needs  is  an  integral  component  of  instruction,  and  it  can 
increase  teaching  effectiveness  significantly.  (Chapter  2) 

Over  the  past  three  years,  there  has  been  increasing  advocacy  for  and  research  showing  the  critical 
importance  of  the  socio-emotional  aspects  in  distance  education  (Xia,  2007),  an  area  long  recognized  in  the 
traditional  classroom  as  a key  to  motivation,  engagement,  satisfaction,  and  positive  interpersonal  interactions 
leading  to  effective  learning  (Main,  1992).  Based  on  research  on  distance  learning  courses,  students  are  seeking 
presence  and  the  immediacy  of  the  face-to-face  classroom  (Perez-Prad  & Thirunarayanan,  2002).  Beyond 
interaction,  social  networking  is  enabling  new  capabilities  to  support  affective  domains  of  learning,  in  part  because 
these  new  tools  make  this  a more  easily  envisioned  and  achievable  goal  (Minocha,  2009).  This  study  is  an 
examination  of  the  premise  that  social  networks  can  enhance  the  affective  aspects  of  learning  in  a distance  education 
class. 


Method 

The  research  design  is  a mixed  methods  case  study  following  Yin  (2003)  in  using  both  qualitative  analysis 
and  rich  description.  The  use  of  case  studies  was  recommended  by  Godwin,  Thorpe,  and  Richardson  (2008)  as  a 
way  to  better  understand  the  high  variance  and  conflicting  results  of  earlier  studies  of  computer-mediated  interaction 
in  distance  education.  In  particular,  the  authors  noted  the  need  to  fully  document  the  specifics  of  design  to  avoid 
over-generalization  as  well  as  point  to  variables  that  potentially  impacted  these  mixed  results. 

This  case  study  involved  two  classes  representing  four  sections  of  an  undergraduate  educational  technology 
course.  The  class  was  an  asynchronous  online  course  offered  in  summer  2008  conducted  in  a six-week  condensed 
format  at  a large  public  research  intensive  university.  No  face-to-face  sessions  were  available,  although  each  class 
had  two  optional  synchronous  sessions  for  students  with  questions.  Only  four  students  total  attended  these  optional 
events.  Each  class  was  taught  by  two  instructors  with  a different  lead  in  each  class.  The  course  is  required  for 
students  in  teacher  certification  programs  and  an  elective  for  graduate  students. 

The  two  classes  included  5 1 students,  with  1 1 males  and  40  females  reflecting  the  common  predominance 
of  females  typical  of  education  courses.  60.1%  were  in  teacher  certification  programs  with  23  undergraduates  and  8 
post-baccalaureate.  The  remaining  19  students  were  in  graduate  programs  or  were  practicing  teachers  taking  summer 
courses  for  professional  development.  For  seventeen  students,  the  class  represented  their  first  online  course 
experience,  with  twelve  more  having  taken  only  a single  one.  The  class  represented  a range  of  age  groups,  with  27 
students  in  their  early  twenties  and  the  rest  older,  including  five  students  over  45. 
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Data  for  this  analysis  came  from  multiple  sources,  including  course  materials  and  teacher  field  notes, 
student  artifacts,  and  student  achievement  via  final  grades.  48  students  completed  an  end-of-course  evaluation  which 
included  open-ended  questions  about  the  course  and  the  use  of  social  networking.  Analysis  included  both  statistical 
and  qualitative  coding.  Data  on  their  use  and  interactions  was  combined  with  post-course  evaluation  comments  to 
analyze  levels  of  interaction,  tool  use,  satisfaction,  and  impact  on  actual  and  perceived  learning.  Analysis  included 
coding  and  thematic  analysis  of  textual  comments  on  evaluations  and  social  network  analysis  of  interaction  patterns 
recorded  within  the  course  Ning  site. 


The  Case  Setting 

As  part  of  an  iterative  design  process  which  has  been  ongoing  over  four  years  to  improve  student  evaluations 
and  satisfaction  with  a required  educational  technology  course  for  preservice  teachers,  a social  networking  tool, 
“Ning,”  was  incorporated  as  a central  component  of  summer  classes.  Initiated  when  a comparison  of  face-to-face 
versus  online  classes  showed  consistently  lower  student  satisfaction  ratings  on  evaluations,  even  with  the  same 
content  and  instructors,  the  course  has  become  a laboratory  for  an  ongoing  design  research  study  on  the  impact  of 
tools  and  strategies  on  student  learning,  attitudes,  and  engagement  in  online  learning  (E.  Hoffman,  2008;  E. 

Hoffman  & Menchaca,  2008;  E.  Hoffman,  et  al.,  2008;  E.  S.  Hoffman,  2009;  Menchaca  & Hoffman,  2009). 

This  course,  “Computers  in  Education,”  evolved  from  earlier  F2F  and  text  based  online  versions  developed 
by  other  instructors,  with  major  online  revisions  in  2006  and  2008.  Earlier  research  findings  showed  that  increasing 
instructor  presence  had  a positive  impact  on  student  end-of-course  evaluations,  raising  the  ratings  on  overall 
satisfaction  and  teacher  factors  to  levels  comparable  to  the  face-to-face  versions  (E.  Hoffman  & Menchaca,  2008). 
But  students  continued  to  write  that  they  missed  the  interaction  of  the  classroom  and  either  preferred  or  had  no 
preference  in  selecting  face-to-face  over  in-person  classrooms.  Few  students  said  they  would  prefer  a DL  option  if 
they  had  a choice.  Discussion  boards  and  group  projects  helped  ensure  both  student- student  and  teacher-student 
interactions,  and  these  were  viewed  positively.  Based  on  student  comments,  still  missing  was  the  immediacy  of 
talking  and  informal  interactions  that  made  classmates  “real,"  par  alleling  earlier  studies  suggesting  teacher  presence 
is  easier  to  establish  than  student  presence  (Russo  & Campbell,  2004).  These  absences  are  the  spaces  where  students 
informally  note  the  weather,  learn  about  what  happens  beyond  the  classroom,  comment  on  the  day-to-day  events  in 
school,  view  dress  and  demeanor,  and  even  commiserate  on  the  level  of  homework  assigned,  establishing  a positive 
affective  learning  environment.  Such  conversations  and  interactions  beyond  content  are  part  of  the  community 
building  that  have  been  more  difficult  to  approximate  in  asynchronous  classrooms. 

The  course  had  been  designed  using  the  institution’s  course  management  system,  which  changed  from 
WebCT  to  a version  of  the  open-courseware  product  Sakai  in  late  2007.  Many  students  had  experience  with  the 
CMS  prior  to  the  educational  technology  class  as  it  is  used  both  in  distance  education  and  as  a supplement  in  face- 
to-face  classes  for  education  students.  In  this  course,  the  CMS  was  used  to  provide  content  through  lesson  modules 
and  assignment  details.  Final  assignments  were  uploaded  through  the  CMS,  and  grades  posted  in  the  online 
gradebook.  The  course  does  not  use  a textbook  with  the  goal  of  keeping  content  highly  current  and  linked  to  real- 
world  examples,  so  all  readings  are  done  online  with  multimedia  and  interactive  components  as  well  as  text  used  in 
presenting  concepts. 

The  course  evolved  both  through  new  tools  and  improved  design,  but  a key  piece  in  recent  years  has  been 
the  ability  to  incorporate  Web  2.0  tools.  In  summer  2008,  the  most  noticeable  change  for  students  was  the  strategic 
placement  of  social  networking  as  a course  tool  with  the  addition  of  Ning.  Ning  was  the  space  for  required 
asynchronous  discussions  rather  than  in  the  CMS.  This  particular  tool  was  selected  because  it  was  more  bounded 
than  the  major  public  sites  like  Facebook  and  MySpace,  allowing  for  creation  of  a private  class-member  only 
network.  Further,  Ning  includes  a threaded  Forum  tool  which  not  only  presents  a student's  response  but  also  a 
thumbnail  picture  each  time  they  post,  a means  of  continually  reminding  students  that  the  respondent  was  a real 
person.  From  a design  perspective,  a social  networking  tool  met  needs  for  a more  comfortable,  user-centered  space 
for  student-student  interaction  than  was  found  in  the  institutionally  provided  course  management  system.  Ning  was 
particularly  useful  for  posting  student-contributed  files,  images,  and  multimedia.  The  course  Ning  became  a model 
for  the  use  of  emerging  collaboration  technologies  to  support  learning  objectives  while  injecting  a space  that  was 
familiar  to  the  “digital  natives”  making  up  not  only  half  the  course’s  students,  but  all  the  children  these  future 
teachers  will  encounter  in  K-12  schools. 

In  the  first  week  of  class,  students  were  required  to  join  the  course  Ning,  create  their  profile  page  including 
posting  a picture  of  themselves,  and  encouraged  to  answer  some  questions  on  their  profile  page  to  introduce 
themselves  to  their  classmates.  They  were  informed  that  the  Ning  was  limited  to  class  members-only  so  that  privacy 
was  assured.  No  tutorials  were  provided  although  the  course  module  linked  to  help  documents  found  elsewhere 
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online  for  those  who  wanted  additional  information.  The  only  initial  problems  reported  were  technical  related  to 
registering  for  a Ning  account,  suggesting  students  quickly  figured  out  the  interface.  Students  were  then  required  to 
respond  to  a Forum  discussion  question  in  Ning  introducing  themselves  as  a way  to  ensure  each  understood  the  tool 
which  was  later  used  for  both  required  discussions  and  sharing  assignment  products  such  as  documents,  useful  Web 
sites,  and  lesson  plans.  The  course  design  included  12  required  discussions  over  the  six  week  period  in  Ning,  with 
half  of  these  requiring  not  only  a response  to  the  posted  question  but  a reaction  or  reflection  to  other  students’ 
posting.  When  not  required,  students  had  the  choice  of  reacting  to  other’s  answers. 

Other  features  of  Ning  were  offered  as  options  but  not  required  during  the  course.  The  instructors 
periodically  posted  pictures  and  videos,  created  blog  entries  which  included  teachers  from  local  schools  sharing  their 
experiences,  and  posted  non-required  discussions  to  share  information  and  useful  Web  resources.  Built-in  features 
always  evident  to  students  logging  in  were  member  pictures,  indications  of  who  was  online,  and  updates  to  what 
recent  activities  other  students  had  done  in  Ning. 

Student  Actions  and  Reactions 

Results  from  end-of-course  student  evaluations  indicated  that  all  but  one  student  were  highly  satisfied  with 
using  Ning.  In  coded  results  from  open-ended  responses  about  their  reactions  to  the  course  overall,  almost  every 
response  was  related  to  the  social  network  components.  Eighteen  mentioned  the  ability  to  get  to  know  the  other 
students  and  17  indicated  they  like  the  level  of  interaction,  with  over  half  indicating  it  was  enjoyable  and  fun.  In 
fact,  several  said  the  social  connections  established  were  better  than  in  traditional  classrooms,  including  these: 

I loved  the  personal  piece  of  it.  I enjoyed  learning  from  my  peers  and  seeing  what  works  for  them  in  their 
classroom.  I also  liked  that  I really  got  to  know  those  students  with  whom  I had  more  in  common  with.  It 
was  easy  to  communicate  and  nice  to  learn  about  everyone  in  the  class,  whom  I may  not  have  interacted 
with  in  a face-to-face  class. 

[Ning]  has  various  meaningful  functions  that  can  replace  traditional  classroom  environment.  It  makes 
teaching  and  learning  activities  more  collaborative,  interesting  and  enjoyable. 

Students  were  satisfied  with  the  ability  to  discuss  ideas  in  class  and  the  intensity  of  interaction  that 
developed.  There  was  an  appreciation  of  differences  and  a strong  sense  of  personalization.  Students  wrote: 

The  best  thing  I like  about  the  Ning  experience  is  the  discussion  activity  with  the  use  of  technology.  I think 
that  it  helps  stimulate  ideas  and  develop  better  communication  skills  to  diverse  group  of  individuals  that  I 
hadn't  fully  known  or  met  but  just  through  pictures  and  exchange  of  messages  or  ideas. 

I am  on  Facebook  and  Myspace,  so  I enjoyed  fixing  up  my  page  to  show  my  personality.  I also  think  it 
forced  the  class  to  interact  and  learn  more  about  each  other  which  is  a great  thing. 

I liked  that  we  could  be  ourselves  in  it. 

The  single  area  of  complaint  was  that  a lack  of  a tool  integrated  within  the  CMS  for  discussions  was 
confusing  to  some,  particularly  noted  by  a few  with  experience  in  previous  self-contained  WebCT  courses.  Four 
students  indicated  that  the  need  to  open  an  external  website  and  log  in  separately  from  the  CMS  required  an 
adjustment  at  first,  but  that  this  was  a short  term  reaction.  All  who  expressed  concern  said  they  were  very 
comfortable  with  the  use  of  Ning  after  the  first  week. 

From  a content  learning  perspective,  there  was  little  evidence  to  support  the  use  of  this  social  networking 
experience  as  one  that  directly  enhanced  student  learning.  Student  performance  on  required  discussions  paralleled 
that  of  previous  classes  that  did  not  use  Ning.  There  was  variability  in  length  and  quality  of  discussion  responses, 
typically  matched  by  the  performance  on  other  assignments.  Despite  self-reports  of  increased  learning,  instructor 
assessments  suggested  that  the  level  of  learning  was  not  impacted  by  the  interaction  and  postings  using  Ning  beyond 
what  could  have  been  done  in  other  discussion  boards.  Because  these  students  take  only  one  course  in  educational 
technology,  there  was  no  easy  way  to  determine  if  indeed  an  individual’s  performance  was  actually  better  than 
might  have  been  predicted  without  social  networking.  Typically  students  do  well  in  the  class  and  this  was  continued 
in  the  summer  2008  iteration.  Retention  was  excellent,  with  only  a single  student  not  completing  all  the  required 
assignments,  a rate  better  than  in  the  typical  educational  technology  F2F  classes.  Some  students  noted  that  the 
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flexibility  of  an  asynchronous  class  made  it  easier  to  deal  with  issues  outside  of  school  that  might  have  impacted 
their  ability  to  attend  classes  and  meet  F2F  deadlines.  However,  the  contribution  of  social  networking  to  high 
retention  is  unknown  as  other  factors  appear  to  have  impacted  the  success  rate,  with  course  structure  and  delivery 
mechanisms  as  more  likely  contributors. 

One  aspect  of  variation  that  emerged  in  the  content  analysis  on  discussion  postings  when  examining 
differences  between  the  two  classes  was  the  level  of  influence  a first  posted  response  had  on  subsequent  answers  by 
the  rest  of  the  class.  When  the  first  response  was  in-depth  and  longer,  all  the  following  responses  tended  to  follow 
this  pattern.  In  most  cases,  first  responses  to  a discussion  were  posted  by  students  who  consistently  responded  early 
and  in  general,  showed  high  achievement  on  other  assignments  in  the  class.  These  high  quality  responses  appeared 
to  trigger  more  critical  thinking  even  from  students  who  were  lower  performing  on  other  tasks.  However,  when  the 
first  response  came  from  a less  high  achieving  student,  lacking  the  level  of  critical  thinking  and  analysis  about  the 
question,  all  following  responses  were  weaker,  even  from  the  top  students.  While  students  had  a rubric  for  how 
discussions  were  to  be  assessed,  and  in  early  discussions  were  provided  with  feedback  to  help  improve  the  content 
of  responses,  the  students  were  more  sensitive  to  what  others  posted  than  some  more  objective  quality  indicators. 

Analysis  of  non-required  contributions  to  the  Ning  discussion  Forum  showed  a mix  of  short  comments  of 
support  for  ideas  and  actions,  including  statements  of  appreciation  for  responses  that  were  insightful  or  helpful.  All 
but  four  students  did  this  at  least  once  during  the  semester,  with  the  non-contributors  or  “lurkers”  being  students 
who  in  general  did  the  minimum  required  on  assignments.  Active  social  use  versus  lurking  did  not  vary  by  age,  nor 
by  previous  experience  with  social  networking  or  distance  education  except  in  a single  case  of  one  older  student 
who  indicated  she  was  unwilling  to  share  personal  information.  At  the  same  time,  she  liked  the  level  on  interaction 
in  discussions  and  left  comments  on  others’  profiles. 

One  of  the  most  notable  uses  in  student-to-student  interactions  in  the  discussion  Forums  were  the  students 
who  posted  extensive  tips  to  help  others  with  assignments,  shared  useful  tools  they  found,  or  provided  technical 
assistance  to  anyone  experiencing  difficulties  with  course  tools.  About  a quarter  of  the  students  took  on  this  role 
during  the  semester,  providing  a level  of  voluntary  peer  tutoring  that  was  unique  to  these  two  classes,  went  far 
beyond  help  usually  seen  in  the  technology  course,  and  which  clearly  contributed  to  smoothing  the  way  for  more 
novice  technology  users  to  be  successful  and  less  stressed  in  the  class.  Many  students  commented  on  how  helpful 
the  sharing  was  overall  in  the  course: 

[Ning]  was  new.  I enjoyed  all  the  different  things  that  we  are  able  to  do  on  it,  such  as:  commenting  each 

other,  posting  discussions  and  commenting  on  them,  being  able  to  look  at  classmates'  examples  for  less 

confusion,  etc. 

While  all  found  Ning  easy  to  use,  there  was  a marked  generational  difference  in  the  means  for  sharing  of 
optional  personal  information  that  is  a fundamental  aspect  of  social  networking  interactions  in  tools  like  Facebook 
or  MySpace.  Students  in  their  twenties,  most  of  whom  indicated  familiarity  with  social  networking,  were  more 
likely  to  produce  long  individual,  text-based  narratives  describing  themselves  and  their  lives  in  great  detail.  By 
contrast  older  students  tended  to  write  few  personal  details  but  readily  shared  photos  of  family,  children  and  pets. 
These  latter  students  typically  had  not  previously  used  sites  like  Facebook  or  MySpace.  Students  regularly  posted 
personal,  non-course  related  comments  on  each  other’s  profile  pages  but  this  behavior  generally  decreased  over  the 
six  weeks  of  the  course  as  the  assignments  became  more  rigorous.  Students  had  from  five  to  31  comments  on  their 
pages  by  the  end  of  the  semester,  with  an  average  of  10.1  per  student.  Interestingly,  most  comments  were  from 
different  individuals  with  no  more  than  five  comments  on  one  student’s  page  coming  from  a single  person.  All  but 
four  students  posted  a profile  comment  at  least  once.  The  “friend”  comments  in  the  later  part  of  the  semester  often 
centered  around  the  two  group  project  assignments  where  such  personal  comments  continued.  Since  groups  were 
assigned,  this  friendly  banter  and  praise  was  generally  among  students  who  had  been  strangers  when  the  semester 
started.  In  fact,  in  many  cases,  the  students  involved  were  geographically  separated  and  would  have  had  no  contact 
outside  of  class.  As  one  student  noted,  virtual  communication  continued  beyond  the  classroom  in  some  cases: 

I really  liked  the  Ning.  I feel  like  I made  friends  through  the  Ning  and  actually  spend  quite  a bit  of  time 

outside  of  class  (chat  and  email)  with  people  I met.  We  plan  to  stay  in  touch! 

In  open-ended  questions  on  the  course  evaluations,  students  consistently  indicated  that  the  interactions  with 
students  in  Ning  positively  impacted  their  perceived  learning  and  enjoyment  of  the  course.  All  but  three  students 
echoed  this  broad  theme  in  unprompted  short  answers  in  which  they  could  have  written  about  any  aspect  of  the 
course.  The  most  common  sub-themes  were  related  to  affective  aspects  supported  through  social  networking. 
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Students  specifically  commented  on  increased  comfort  in  working  with  others  they  “knew,”  the  interactions  that 
went  beyond  required  course  discussions,  and  the  impact  of  “knowing”  that  helped  as  they  worked  together  on 
assignments.  Many  suggested  Ning  made  the  course  fun  and  encouraged  them  to  check  in  to  see  what  others  were 
doing,  suggesting  it  led  to  a high  level  on  engagement. 

I really  loved  it.  It  was  like  WebCT,  but  it  seemed  more  social  and  friendlier  to  work  with.  It  made  me  want 
to  log  onto  it  because  it  "tricked"  me  into  feeling  like  it's  not  school. 

Other  themes  on  open-ended  comments  at  the  end  of  the  course  included  that  Ning  increased  and  improved 
peer  feedback,  made  others  more  visible,  included  multimedia  capabilities,  and  allowed  for  creativity.  A number 
who  had  taken  previous  distance  classes  noted  how  much  they  liked  this  course’s  design  over  previous  examples. 
From  an  instructor  perspective,  the  use  of  Ning  did  not  impact  assessed  learning  outcomes  but  did  increase  student 
self-support  to  figure  out  course  requirements.  At  the  end  of  the  course,  two-thirds  of  the  students  indicated  that  they 
believed  distance  learning  was  as  good  as  or  equal  to  learning  in  face-to-face  classes,  a significant  change  from 
evaluations  in  earlier  classes. 


Conclusion 

Many  of  the  findings  from  this  research  parallel  that  of  earlier  case  studies  of  distance  learning,  including 
the  impact  of  grading  on  participation  and  the  expressions  of  need  for  interaction  and  social  presence.  In  examining 
the  impact  of  the  use  of  a social  network  in  an  asynchronous  class,  there  was  little  evidence  in  this  case  study  to 
support  the  idea  that  it  led  to  improved  student  learning  as  measured  by  grades,  outcomes  on  assignments,  or  peer 
assessments  done  as  part  of  the  course  work.  In  general,  earlier  improvements  related  to  instructional  strategies  and 
design  reported  in  previous  design-research  studies  about  the  course  had  greater  impact  than  the  use  of  Ning.  While 
self-reports  of  learning  by  the  students  in  the  summer  class  indicated  they  believed  they  had  been  more  successful, 
objective  evidence  was  not  there  to  support  their  views.  They  were  highly  successful  but  no  more  than  in  previous 
classes. 

However,  the  results  are  more  clear  when  it  comes  to  issues  of  motivation,  satisfaction,  and  engagement. 
These  students  were  ecstatic  and  uniform  in  describing  their  experiences  in  the  class  related  to  what  happened  in 
Ning.  While  some  commented  on  the  modules  or  quality  of  assignments,  every  student  was  positive  about  some 
aspect  of  social  networking.  They  were  particularly  pleased  with  the  connections  they  made  to  the  other  students  in 
the  asynchronous  class.  None  had  complaints  other  than  over  initial  log  in  confusion  which  was  quickly  cleared  up. 
Further,  they  were  energized  about  the  possibility  of  future  DL  classes. 

Two  areas  of  interest  for  future  research  emerged.  In  the  first,  despite  many  popular  works  about 
differences  between  digital  natives  and  digital  immigrants,  there  was  no  difference  in  terms  of  satisfaction  or 
concerns.  A unique  finding  was  the  difference  by  age  grouping  in  the  use  of  personal  sharing  mechanisms,  with 
more  text  presented  by  younger  students  and  more  multimedia  by  older  ones.  Students  propensity  for  socializing 
varies  along  other  dimensions,  possibly  personal  preferences  or  learning  styles,  which  remains  a question  for  future 
research.  Lurkers  seem  to  be  equally  successful  and  satisfied,  making  many  current  measures  of  engagement  in  DL 
harder  to  link  to  outcomes.  The  second  area  had  to  do  with  the  social  networking  impact  on  the  high  level  of 
voluntary  peer-helping  behaviors,  particularly  for  technical  and  resource  sharing.  Both  of  these  are  now  being 
examined  in  the  context  of  a summer  2009  version  of  the  course,  but  at  this  time  analysis  is  incomplete. 

This  case  study  has  pointed  to  the  importance  of  affective  domains  which  can  make  an  online  course  seem 
less  stressful,  pleasurable,  and  worth  attending  and  completing.  These  affective  elements  were  reflected  primarily  in 
fewer  complaints  during  the  semester  and  in  self-reports  at  the  semester’s  end.  Ning  made  it  possible  to 
accommodate  individual  needs  for  personalizing  a course  and  creating  an  environment  in  which  student  presence 
becomes  “comfortable.”  To  the  extent  that  student  satisfaction  leads  to  a continued  desire  to  learn  and  greater 
comfort  with  new  learning  delivery  mechanisms,  adding  a social  dimension  to  the  class  was  a low-cost  effort  that 
had  a high  emotional  payoff  as  shown  in  this  study.  With  new  tools  readily  and  freely  available,  instructors  and 
designers  have  increased  options  for  adding  an  explicitly  affective  component  to  online  courses  if  they  can  move 
beyond  the  cognitive  concerns  more  typical  in  higher  education  teaching. 

In  their  recent  popular  book,  “Disrupting  Class,”  Christensen,  Horn,  and  Johnson  (2008)  called  for  more 
student-centered  learning  and  argued  that  emerging  technologies  now  appearing  will  cause  disruptive,  revolutionary 
changes  to  the  education  enterprise.  However,  in  reviewing  the  changing  landscape  of  online  interaction  in  the 
design  research  process  for  this  course,  what  emerges  is  the  potentially  more  gradual  shift  to  personal  learning 
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environments  allowing  individual  preferences  and  expressions.  The  use  of  social  networking  tools  allows 
evolutionary  changes  within  existing  structures  and  known  pedagogies  that  result  in  students  who  are  more  engaged 
and  excited  by  learning. 
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Short  Description: 

This  is  a demonstration  of  a system,  including  a web-based  self-regulated  learning  (SRL)  tutorial  and 
strategy  application,  to  facilitate  SRL  in  the  online  environment.  It  was  designed  based  on  learning  strategy 
development  literature  to  provide  modeling  and  self-explanation,  self-reflective  practice  and  feedback  on  SRL 
strategies,  to  promote  metacognitive  awareness,  and  to  enhance  motivational  processes.  Suggestions  for 
implementation  and  instructional  implications  will  be  discussed.  This  session  may  be  interesting  to  instructors  and 
administrators  of  online  leaning. 


Introduction 

Two  of  the  major  problems  facing  web-based  distance  education  are  high  attrition  rate  and  low  learner 
satisfaction.  Numerous  interrelated  reasons,  such  as  lack  of  time  (Kember,  Murphy,  Siaw,  & Yuen,  1991;  Osborn, 
2001)  and  environment  (Osborn,  2001)  management  skills,  mismatch  between  learners’  interest  and  course  structure 
(Chyung,  2001;  Kember  et  al.,  1991),  and  low  self-motivation  (Osborn,  2001;  Parker,  2003),  are  causing  students  to 
drop  out  of  distant  programs.  A big  portion  of  these  reasons  is  related  with  learners’  academic  self-regulation. 
Several  researchers  (Eom,  1999;  King,  Harner,  & Brown,  2000)  have  identified  self-regulatory  strategies  as 
predictors  for  achievement  in  the  online  mode.  Furthermore,  studies  (Chyung,  2001;  Ulitsky,  2000)  inform  us  that 
adequate  and  appropriate  training  on  self-regulated  learning  strategies  is  able  to  reduce  the  attrition  rate  greatly. 

Zimmerman  (1990)  defines  self-regulated  learning  (SRL)  with  three  distinctive  features:  learners’ 
application  of  self-regulated  learning  strategies,  their  sensitivity  to  self-evaluative  feedback  about  learning 
effectiveness,  and  their  self-generated  motivational  processes.  The  literature  supports  the  notion  that  learners’  self- 
regulation is  a powerful  predictor  for  their  academic  achievement  (Ley  & Young,  1998;  Pintrich  & Groot,  1990).  In 
addition,  self-regulation  of  learning  progress  is  found  to  have  a positive  effect  on  learners’  motivation  (Lan,  1996; 
Zimmerman  & Kitsantas,  1996)  and  use  of  learning  strategies  (Lan,  1996;  Schunk  & Ertmer,  1999). 

New  learning  environments,  such  as  web-based  or  web-enhanced  instruction,  require  more  learner  self- 
control  and  proactive  learning  to  construct  knowledge  and  acquire  skills.  As  Schunk  and  Zimmerman  ( 1998) 
mentioned  that  “an  area  that  lends  itself  well  to  self-regulation  is  distance  learning,  where  instruction  originates  at 
one  site  and  is  transmitted  to  students  at  distant  sites. . . Self-regulation  seems  critical  due  to  the  high  degree  of 
student  independence  deriving  from  the  instructor’s  physical  absence”  (p.  231-232).  Fortunately,  Pintrich  ( 1995) 
pointed  out  that  “self-regulated  learning  is  teachable”  (p.7).  Teachers  can  teach  students  to  become  self-regulated 
learners,  while  students  can  learn  to  be  more  self-regulated. 

This  system  to  be  demonstrated  was  designed  and  developed  with  the  hope  to  provide  online  learners 
training  on  and  practice  of  self-regulated  learning  strategies  in  order  to  reduce  the  attrition  rate  and  increase  learner 
motivation  and  satisfaction. 


Design  Rationale  of  SRfOLTS 

Instructional  design  principles  in  the  learning  strategy  development  literature  served  as  the  rationale  for  the 
design  and  development  of  this  web-based  SRL  strategies  training. 
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Principle  1:  Promote  Learners’  Metacognitive  Awareness  of  Their  Behavior,  Motivation,  and  Cognition. 
First,  metacognitive  awareness  is  the  prerequisite  for  learners  to  be  able  to  learn  from  and  utilize  the  self-regulated 
learning  strategies  that  interventions  are  intended  to  teach.  It  is  suggested  (Pintrich,  1995)  that  standardized 
assessment  instruments,  such  as  the  Motivated  Strategies  for  Learning  Questionnaire  (Pintrich,  Smith,  Garcia,  & 
McKeachie,  1991)  or  the  Learning  and  Study  Strategies  Inventory  (Weinstein,  Husman,  & Dierking,  2000),  can  be 
used  to  provide  students  with  an  overview  of  their  motivational  beliefs  and  learning  strategies. 

Principle  2:  Provide  Strategy  Training  Using  the  Methods  of  Modeling  or  self- 
Constructions/Explanations.  Strategy  teaching  is  regarded  (Schunk  & Zimmerman,  1998)  as  a major  means  of 
promoting  self-regulated  learning.  Modeling  and  self-  constructions/explanations  are  the  two  major  methods  used  to 
teach  self-regulated  learning  strategies. 

Principle  3:  Provide  Ample  Opportunities  for  Learners  to  Practice  Self-Regulated  Learning  Strategies  and 
Feedback  about  Strategy  Effectiveness.  Two  crucial  elements  are  practice  of  self-regulatory  strategies  and  feedback 
on  strategy  effectiveness.  These  mechanisms  facilitate  learning  and  motivation  by  communicating  learning  progress, 
and  they  also  enhance  strategy  transfer  and  maintenance  (Schunk  & Zimmerman,  1998). 

Principle  4:  Incorporate  Motivational  Processes,  Especially  Positive  Beliefs,  into  Instruction.  Motivation 
plays  an  important  role  in  developing  self-regulated  learning.  To  participate  in  self-regulation  requires  that  students 
have  the  motivation  or  willingness  to  learn  over  extensive  periods  (Schunk  & Zimmerman,  1998).  Motivation  is 
essential  because  it  plays  a mediating  role  to  the  effect  of  self-regulated  learning  strategies. 

Principle  5:  Integrate  Self-Reflective  Practice  Systematically  with  Instruction  on  Self-Regulated  Learning 
Strategies.  Finally,  across  interventions  there  is  an  emphasis  on  self-reflective  practice,  which  refers  to  students’ 
practice  of  skills  and  reflection  on  their  performance.  Self-reflective  practice  often  is  built  into  the  instructional 
procedure  with  independent  practice  or  time  for  self-reflection  (Schunk  & Zimmerman,  1998). 

Format  of  Self-Regulated  Learning  Strategy  Training 

In  a learner-centered  environment,  such  as  web-assisted  instruction,  learners  do  not 
automatically  possess  the  metacognitive  skills  required  to  make  independent  judgments  and  selections  about  how  to 
learn  (Brown,  Hedberg,  & Harper,  1994).  However,  Hofer,  Yu,  & Pintrich  (1998)’s  work  suggests  that  an 
intervention  that  targets  a range  of  cognitive  and  motivational  components  can  benefit  college  students  who  need 
help  with  metacognitive  skills.  In  addition,  Hofer  claims  that  there  is  a need  for  adjunct  interventions  that  stress 
general  strategies  for  at  least  some  college  students  and  a need  for  research  on  how  to  design  adjunct  courses 
regarding  the  tasks  and  instructional  strategies. 

Components  of  SRfOLTS 

Based  on  the  Instructional  design  principles  in  the  learning  strategy  development  literature,  this  SRfOLTS 
is  made  up  of  three  components:  1)  an  online  administration  of  the  Motivated  Strategies  for  Learning  Questionnaire 
(MSLQ)  (Pintrich  et  al.,  1991)  for  pre-and  after-intervention  assessment,  2)  an  adjunct  web-based  online  tutorial 
(Hofer  et  al.,  1998),  and  3)  self-reflective  practice  (Graham,  Harris,  & Troia,  1998;  Schunk  & Zimmerman,  1998) 
using  online  forms/questionnaires. 

The  intervention  is  a semester-long  (Ertmer  & Newby,  1996;  Pintrich,  1995)  multi-strategy  program  (Hofer 
et  al.,  1998)  that  teaches  a range  of  strategies  for  students  to  have  both  the  “skill”  and  the  “will”  to  use  the  strategies 
properly.  The  training  tried  to  utilize  the  methods  of  modeling  (Bandura,  1986;  Kitsantas,  Zimmerman,  & Cleary, 
2000)  and  self-explanation  (Bielaczyc,  Pirolli,  & Brown,  1995)  through  video  with  real  student  example  and  ample 
opportunity  for  exercise  and  feedback  (Ertmer  & Newby,  1996;  Pintrich,  1995)  through  interactive  case  study  with 
automatically-generated  feedback  on  knowledge  of  strategies,  and  online  Study  Plan  & Self-Evaluation  as  practice 
for  application  of  strategies  in  completing  assignments.  Every  step  of  the  intervention,  e.g.  the  prior  questions,  the 
real-world  cases  in  tutorial,  the  MSLQ,  served  the  purpose  for  promoting  learners’  metacognitive  awareness  (Hattie, 
Biggs,  & Purdie,  1996;  Pintrich,  1995).  Motivational  issues  (Pintrich,  1995)  are  addressed  (e.g.,  building  confidence 
through  moving  from  knowledge  to  application  of  strategies,  pintable  job  aids,  printable  certificate  for  completion  of 
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tutorial)  throughout  the  whole  intervention.  Learners  are  assessed  on  their  prior  and  post  motivational  beliefs  and 
self-regulatory  approaches  using  the  MSLQ,  and  a tutorial  is  used  to  facilitate  students’  revision  of  problematic  self- 
regulated  learning  process. 

SRL  Tutorial 

This  web-based  tutorial  provide  students  with  both  the  knowledge  and  practice  on  knowledge  of  self- 
regulated  learning  strategies  through  interactive  exercises,  such  as  multiple-choice  or  matching,  and  a test. 

This  tutorial  focused  on  what  SRL  strategies  are,  more  specifically,  what  metacognitive  (planning, 
monitoring,  evaluating),  motivational  (goal  orientation,  self-efficacy),  resource-management  strategies  (time 
management,  help  seeking)  cognitive  strategies  (rehearsal,  elaboration,  organization)  are,  as  well  as  examples  of 
them.  In  addition  to  different  types  of  strategies,  this  tutorial  also  discussed  when  and  how  to  use  them  (Hattie  et  al., 
1996),  which  enables  the  students  to  make  use  of  the  strategies  effectively. 

Strategy  Application  Practice 

After  they  exercise  on  the  knowledge  about  SRL,  the  learners  are  encouraged  to  actually  apply  the 
strategies  to  their  studying  of  the  academic  content  of  a particular  course.  A number  of  online  forms/questionnaires 
were  developed  to  implement  this  practice,  and  they  can  be  sent  to  the  participants  as  a link  in  emails.  These  online 
forms  can  be  delivered  as  a set  at  the  beginning  (for  strategic  planning)  and  end  (for  self-evaluation)  of  each  learning 
period  of  4 weeks.  A total  of  2 sets  of  online  questionnaires  can  be  used  during  the  process  of  study  for  strategy 
practice.  The  following  is  a description  of  how  the  strategies  were  represented  in  the  online  forms/questionnaires  for 
learners  to  apply  them. 

Goal  setting  is  defined  as  students’  choice  of  desired  standard  for  learning  or  performance.  Students  are 
asked  “What  are  your  goals  (with  respect  to  this  class)  for  this  semester?”  to  indicate  their  choice  of  desired  standard 
for  learning  or  performance  and  they  were  reminded  to  make  sure  their  goals  were  in  line  with  their  general  system 
of  values.  And,  more  specifically,  students  were  asked  to  clarify  their  immediate  objective  by  answering  a question 
“What  tasks  are  you  planning  to  accomplish  in  the  next  two  weeks  to  help  you  achieve  your  goals  for  this  course?” 

Strategic  planning  refers  to  students’  identification  of  strategies  and  allocation  of  resources  for  achieving 
their  goals.  Students  are  provided  with  example  of  strategies,  and  encouraged  to  formulate  their  own  cognitive  and 
motivational  strategies  for  studying.  They  are  asked  “What  will  you  do  to  attain  these  goals  (for  the  next  two 
weeks)?”  They  were  asked  to  provide  a specific  plan  of  action.  Students  were  also  asked  to  describe  how  they  would 
allocate  their  resources.  They  were  asked  questions  such  as  “When  will  you  start  working  toward  these  goals?  On 
what  days?  At  what  times?  How  often?  How  much  time  do  you  intend  to  spend  on  these  tasks?”;  “Where  will  you 
do  these?”;  “With  whom  will  you  do  these?”  and  “What  additional  information/assistance  will  you  need?”  and  so  on. 

Self-evaluation  is  the  opportunity  for  students  to  analyze  their  performance  and  strategy  effectiveness  after 
a period  of  learning.  In  this  system,  it  is  implemented  as  an  online  questionnaire  that  allowed  students  to  self-judge 
their  performance  and  the  progress  that  they  have  made  for  a particular  study  period  of  4 weeks.  Students  are 
presented  with  their  self-defined  individual  goals  and  asked  questions  “How  well  did  your  plan  work  during  these 
past  2 weeks?”  and  “Did  you  complete  the  tasks  you  had  planned  during  these  past  2 weeks?”  They  are  encouraged 
to  give  specific  examples  using  a drop-down  menu  with  choices  of  1)  Improved  test  scores  2) All  homework  turned 
in  3)  Assignments  completed  on  time  4)Readings  completed  5 (Notes  taken  for  chapters  assigned  6)  Other,  please 
explain.  In  addition,  students’  strategy  effectiveness  is  assessed  using  an  open-ended  question  “What  learning 
strategies  worked  well?”  and  their  self-satisfaction  is  assessed  using  multiple  choice  questions,  such  as  “You  feel 
pleased  with  your  progress  in  the  course”,  asking  them  to  rate  their  feelings  on  a 5 point  Likert-type  scale,  ranging 
from  “Not  at  all  true  of  me”  to  “Very  true  of  me”. 

Suggested  Implementation  Procedure 

This  system  can  be  implemented  in  4 stages,  including  soliciting  participation  and  collecting  initial  data  on 
strategy  and  motivation  ( 1 week),  completing  web  tutorial  on  SRL  (4  weeks),  completing  online  study  plan  and  self- 
evaluation  (8  weeks),  and  final  strategy  and  motivation  assessment  (1  week),  and  last  for  about  14  weeks. 
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Instructional  Implications  of  the  SRfOLTS 


While  web-based  or  web-assisted  instructions  are  widely  used  (Warburton,  Chen,  & Bradburn,  2002)  these 
days,  it  is  still  difficult  for  many  students  to  manage  their  learning  in  this  kind  of  environment.  The  SRfOLTS  can  be 
integrated  into  any  online  or  web-assisted  courses  for  college  instructors  to  facilitate  students’  learning.  This  system 
can  also  be  used  with  orientation  for  online  students  to  help  them  evaluate  whether  online  is  an  appropriate  format 
for  them  and  how  they  can  prepare  themselves  with  the  strategies  for  online  learning. 
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Introduction: 

This  literature  review  first  discusses  an  existing  problem  with  community  college  student  success,  a high 
incompletion  rate.  Then,  it  covers  the  theories  and  research  about  SRL  and  learning  strategy  development.  SRL 
strategy  training  has  been  found  effective  to  promote  learning  achievement,  motivation  and  strategy  use.  It  was 
proposed  that  using  SRL  strategy  training,  implemented  in  the  format  of  remedial  or  study  skills  courses,  might  be  a 
solution  to  community  college  student  success. 

Issues  in  Community  College  Student  Success 

Community  college  is  an  important  channel  to  provide  postsecondary  education  in  the  United  States.  It  has 
a large  enrollment  and  is  considered  an  affordable  means  for  educational  attainment.  According  to  a National  Center 
for  Educational  Statistics  report,  community  college  students  represent  about  40%  of  all  undergraduates,  or  about 
7.6  million  students  in  the  whole  U.  S.  (Horn  & Nevill,  2006).  Yet,  student  retention  in  community  colleges  is  a big 
concern  for  many  educational  researchers  and  administrators. 

A problem  of  high  attrition  rates  has  been  discovered  along  with  the  lower  expense  and  open-access 
policies  at  community  colleges.  According  to  studies  conducted  by  the  U.  S.  Department  of  Education  and  some 
other  institutions,  about  35-44  percent  of  college  students  who  began  at  2-year  institutions  dropped  out  without 
completion  (Bradburn  & Carroll,  2002;  Cofer  & Somers,  2000;  Hoachlander,  Sikora,  & Horn,  2003).  Most 
nonpersisters  left  during  the  first  year  of  enrollment  (Bradburn  & Carroll,  2002). 

Students’  drop-out  of  community  colleges  have  close  correlation  with  numerous  reasons,  such  as:  students’ 
demographic  characteristics,  including  age,  family  responsibilities,  working  full-time,  part-time  enrollment,  their 
insufficient  prior  educational  preparation  (Hoachlander  et  al.,  2003;  Horn  & Nevill,  2006;  Wirt  et  al.,  2004), 
unsatisfactory  prior  achievement  in  colleges  (Bradburn  & Carroll,  2002;  Cofer  & Somers,  2000;  Hoyt,  1999)  and 
lack  of  learning  strategies  (Byrd  & MacDonald,  2005;  Ley  & Young,  1998;  Schmid  & Abell,  2003). 

Need  for  College  Success  Courses  and  Study 

Because  of  their  nontraditional  demographic  characteristics  and  educational  preparation,  there  is  a higher 
tendency  for  community  college  students  to  leave  without  a credential  after  they  begin  their  postsecondaiy  education. 
What  can  be  done  to  help  solve  this  issue?  Compared  with  the  large  number  of  freshmen,  minority,  and  first 
generation  students  who  are  entering  community  colleges,  not  enough  research  has  been  conducted  to  understand 
and  help  this  population  (Wild  & Ebbers,  2002). 
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Remedial  or  study  skills  courses  can  assist  in  providing  underprepared  students  with  math,  reading,  English, 
and  study  skills  to  succeed  in  college  (Byrd  & MacDonald,  2005),  to  enhance  achievement  (Stovall,  2000;  Tuckman, 
2002,  2003a,  2003b),  and  to  improve  retention  (Cofer  & Somers,  2000;  Derby  & Smith,  2004;  Stovall,  2000; 
Tuckman,  2003b).  As  a result,  several  researchers  (Byrd  & MacDonald,  2005;  Hoyt,  1999;  Schmid  & Abell,  2003) 
emphasize  the  need  for  interventions  that  concentrate  on  the  academic  needs  of  students  and  call  for  the  use  of 
remedial  education  and  study  skills  courses  to  help  students  understand  learning  strategies,  to  appreciate  the 
importance  of  consistent  studying  rather  than  cramming  (Schmid  & Abell,  2003),  and  to  assist  colleges  in  teaching 
students  skills  for  time-management,  focusing  on  goals,  and  self-advocacy  explicitly  in  their  first  year  of  college 
(Byrd  & MacDonald,  2005). 

Many  of  these  remedial  or  study  skills  interventions  are  designed  and  developed  on  the  basis  of  the  theories 
and  research  on  academic  self-regulation  (Shafer,  Lahner,  Calderone,  Davis,  & Petrie,  2002;  Tuckman,  2003a),  yet 
how  to  make  these  courses  more  effective  and  efficient  at  community  colleges  is  an  area  awaits/worth  investigation. 
This  literature  review  attempted  to  examine  the  connection  between  self-regulated  learning  strategy  and  Community 
College  Success,  and  to  explore  the  implications  this  connection  has  on  instruction.  This  coincides  with  Young  & 
Ley  (2003)’ s recommendation  for  research  on  the  function  of  self-regulated  learning  support  in  remedial  education, 
using  either  qualitative  or  quantitative  methods. 


Theoretical  Framework  about  Self-Regulated  Learning 

The  core  of  college  success  courses  is  the  theory  and  research  results  about  self-regulated  learning.  This 
following  section  introduces  concepts  about  self-regulated  learning  and  presents  research  on  self-regulated  learning 
strategies  and  development  of  self-regulated  learning. 

What  is  Self-Regulated  Learning? 

Generally  speaking,  self-regulated  learning  is  a multidimentional  construct,  which  involves  students’  self- 
generated application  or  adjustment  of  motivation,  cognition,  and  metacognition  (Alexander,  1995)  for  achieving 
optimal  performance  outcome.  Every  learner  is  self-regulated  for  academic  learning  to  some  degree  (Zimmerman, 
1998),  but  there  are  significant  differences  among  students.  Self-regulated  learning  can  be  developed  through 
personal  discovery  or  training  (Ertmer  & Newby,  1996;  Zimmerman,  2000). 

Zimmerman  & Martinez-Pons  (1990)  defines  self-regulated  learning  (SRL)  with  three  distinctive  features: 
learners’  application  of  self-regulated  learning  strategies,  their  sensitivity  to  self-evaluative  feedback  about  learning 
effectiveness,  and  their  self -generated  motivational  processes.  They  differentiate  academic  self-regulation  from 
mental  ability,  such  as  intelligence,  or  an  academic  skill,  such  as  reading  proficiency.  Zimmerman  (1998)  suggests 
that  it  is  a “self-directive  process  through  which  learners  transform  their  mental  abilities  into  academic  skills”  (p.  2). 

Research  about  Self-Regulated  Learning 

Research  conducted  over  the  last  decade,  in  controlled  settings,  has  consistently  found  a positive 
correlation  between  academic  achievement  (Lan,  1996;  Schunk,  1996;  Zimmerman,  Bonner,  & Kovach,  1996), 
motivation  (Schunk,  1996;  Schunk  & Ertmer,  1999;  Zimmerman  & Kitsantas,  1996,  1999)  and  self-regulated 
learning.  However,  even  though  the  use  of  self-regulated  learning  strategies  is  very  helpful  for  students  to  achieve 
better  learning  and  motivational  results,  it  might  be  intervened  by  some  motivational  characteristics,  such  as  goal 
orientation  (Ames  & Archer,  1988;  Greene  & Miller,  1996),  self-efficacy  (Bouffard-Bouchard,  Parent,  & Larivee, 
1991;  Zimmerman  & Martinez-Pons,  1990),  and  interest  and  valuing  (McWhaw  & Abrami,  2001;  Miller,  Behrens, 
Greene,  & Newman,  1993).  Some  of  these  characteristics,  such  as  self-efficacy  and  goal  orientation,  might  even 
affect  persistence  (Miller  et  al.,  1993). 

Field  tests  of  comprehensive  training  on  self-regulated  learning  strategies  to  students  in  traditional 
classroom  settings  have  found  effects  on  improved  task  performance  (Butler,  1998;  Graham,  Harris,  & Troia,  1998; 
Hofer,  Yu,  & Pintrich,  1998;  Paz,  1999;  Weinstein,  Husman,  & Dierking,  2000)  and  metacognitive  understandings 
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(Butler,  1998;  Ching,  2002),  more  positive  motivation  (Butler,  1998;  Ching,  2002;  Graham  et  al.,  1998;  Hofer  et  al., 
1998;  Weinstein  et  al.,  2000),  more  strategy  use  (Butler,  1998;  Hofer  et  al.,  1998;  Weinstein  et  al.,  2000),  and  even  a 
better  retention  rate  (Weinstein  et  al.,  2000). 

A number  of  studies  measure  the  effectiveness  of  SRL  strategy  training  using  the  e-learning  environment. 

In  these  seven  studies  that  reported  empirical  evaluation  of  this  kind  of  training  in  a field  setting,  five  of  them 
(McKeachie,  Pintrich,  & Lin,  1985;  Shafer  et  al.,  2002;  Tuckman,  2002,  2003a,  2003b)  have  been  found  effective  to 
promote  learning  achievement,  three  of  them  found  positive  effects  on  use  of  strategies  (Hartley,  2001;  Hofer  & Yu, 
2003;  McKeachie  et  al.,  1985),  1 found  positive  effects  on  motivation  (Hofer  & Yu,  2003)  and  one  found  benefit  for 
student  retention  (Tuckman,  2003b). 

Integrating  Self-Regulated  Learning  into  Community  College  Success 

Self-regulated  learning  strategy  training  has  been  found  effective  to  promote  learning  achievement  from 
numerous  studies,  either  in  lab-controlled  experiments  (Lan,  1996;  Schunk,  1996;  Zimmerman  et  al.,  1996),  or 
comprehensive  field  interventions  (e.g.,  (Butler,  1998;  Graham  et  al.,  1998;  Weinstein  et  al.,  2000).  Self-regulated 
learning  strategy  training  might  help  community  college  students  achieve  significantly  higher  average  course 
percentage  score  and  average  scores  for  each  of  the  major  course  activities.  Achievement  is  one  of  the  primary 
causes  to  the  high  impletion  rate  in  community  college  (Bradburn  & Carroll,  2002;  Cofer  & Somers,  2000;  Hoyt, 
1999).  Improving  learning  achievement  creates  the  possibility  for  promoting  learners’  self-efficacy  and  satisfaction, 
and  eventually  reducing  drop  out  rate  and  increasing  cost  effectiveness  of  education  provided  at  community  colleges. 

Providing  comprehensive  training  on  self-regulated  learning  strategies  can  bring  about  more  positive 
motivation  in  terms  of  attribution  (Butler,  1998),  self-efficacy  (Ching,  2002;  Graham  et  al.,  1998;  Hofer  et  al.,  1998; 
Weinstein  et  al.,  2000),  and  interest  and  mastery  goal  (Hofer  et  al.,  1998).  In  addition,  students’  engagement  in  self- 
regulated  learning  behaviors  also  seems  to  bring  about  self-satisfaction  (Zimmerman  & Kitsantas,  1999).  Therefore, 
it  is  reasonable  to  assume  that  students’  task  value,  self-efficacy  intrinsic  goal  and  self-satisfaction  would  be  higher, 
and  their  extrinsic  goal  would  be  lower  if/after  they  participate  in  the  a training  on  self-regulated  learning  strategies. 

Since  motivational  learner  characteristics,  such  as  goal  orientation,  self-efficacy,  and  interest  and  valuing, 
might  further  influence  the  future  use  of  self-regulated  learning  strategies.  Some  of  these  characteristics,  such  as 
self-efficacy  and  goal  orientation,  might  even  affect  persistence.  Improving  learner  motivation,  through  the  use  of 
training  on  self-regulated  learning  strategies,  makes  it  possible  for  promoting  students'  future  use  of  learning 
strategies,  and  may  ultimately  lead  to  a decrease  in  drop  out  rate  at  community  colleges. 

Deficiency  in  learning  strategies  has  been  identified  as  another  major  cause  for  the  high  incompletion  rates 
in  community  college  education.  Providing  comprehensive  training  on  self-regulated  learning  strategies  can  lead  to 
more  strategy  use  (Butler,  1998;  Hofer  et  al.,  1998;  Weinstein  et  al.,  2000),  and  even  a better  retention  rate 
(Weinstein  et  al.,  2000).  Engaging  learners  in  learning  and  implementing  self-regulated  learning  strategies  through 
the  training  might  improve  their  strategy  use.  The  potential  enhancement  in  learners’  strategy  use  might  lead  to 
increase  in  completion  rate. 


Instructional  Implications 

Remedial  or  study  skills  courses,  designed  and  developed  on  the  basis  of  the  theories  and  research  on 
academic  self-regulation  (Shafer  et  al.,  2002;  Tuckman,  2003a),  has  been  found  effective  in  providing 
underprepared  students  with  study  skills  to  succeed  in  college  (Byrd  & MacDonald,  2005),  to  enhance  achievement 
(Stovall,  2000;  Tuckman,  2002,  2003a,  2003b),  and  to  improve  retention  (Cofer  & Somers,  2000;  Derby  & Smith, 
2004;  Tuckman,  2003a).  Therefore,  it  is  logical  for  researchers  (Byrd  & MacDonald,  2005;  Hoyt,  1999;  Schmid  & 
Abell,  2003)  to  call  for  the  use  of  remedial  education  and  study  skills  courses  to  help  students  understand  learning 
strategies  and  improve  achievement  and  retention  (Byrd  & MacDonald,  2005;  Schmid  & Abell,  2003)  in  community 
colleges. 
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Abstract 

Information  technologies  are  rapidly  reshaping  the  world,  but  the  digital  divide  has  been  a growing  concern  both  in 
the  domestic  and  international  discourse.  Generally  speaking,  the  poor  and  the  disadvantaged  tend  to  live  in  the 
remotely  isolated  rural  areas.  Gender  plays  a role  in  this  scenario.  This  paper  provides  a review  of  learning 
technologies  as  empowering  tools  for  rural  low  income  women  from  global  and  domestic  perspectives. 

Introduction 

Though  information  technology  (IT)  is  no  longer  considered  a luxury  in  many  affluent  societies,  the  digital 
divide1  continues  to  be  a major  concern  both  in  domestic  and  international  discourse  (NPA  Report,  2002;  PEW, 
2004).  Statistics  shows  a significant  gap  in  access  and  knowledge  of  IT  among  the  rural,  urban  and  suburban  areas 
where  socio-economic  status,  gender,  and  racial  background  distinguish  such  disparity  (World  Bank,  2002).  Issues 
with  accessing  opportunities  and  resources  relating  to  IT  imply  the  increasingly  marginalization  of  low-income  rural 
mothers.  As  Dean  (2000)  remarks  those  who  with  lack  of  access  to  information  technologies  will  become  the  second 
class  citizens  of  the  future.2 

Rural  low-income  women  are  disproportionately  burdened  with  task  loads,  and  have  the  least  social 
mobility  to  access  resources  and  services  such  as  health  care,  child  care  facilities,  social  supports,  education,  and  job 
opportunities.  Many  government  policies,  including  U.S.  Welfare  Reform  in  the  1990s  intend  to  support  low  income 
mothers’  independence  by  providing  job  and  skill  training,  education  opportunities  and  other  relevant  self-sufficient 
strategies.  Learning  technologies  have  been  deemed  as  powerful  tools  to  transform  many  aspects  of  human  lives 
(Brown,  et  al,  2009).  The  following  are  some  practices  gleaned  from  different  regions  of  the  world. 

A Brief  Review  from  the  Global  Perspective 

At  the  global  level,  in  its  Gender  and  Development  Plan  of  Action,  the  Food  and  Agriculture  Organization 
(FAO)  of  the  United  Nations  recognized  the  power  of  IT  in  transforming  both  natural  and  human  capital 
development.  It  pointed  out  that  changes  could  accelerate  progress  through  gender  equality.  It  calls  for  policymakers, 
practitioners  and  communities  to  give  attention  to  the  risks  and  burden  that  women  bear  and  suggested  that  unless 
women  have  access  and  use  new  technologies,  the  inequality  could  be  exacerbated  (FAO,  2002).  We  reviewed 
several  countries  across  different  continents  regarding  their  conditions  and  practices. 

International  Labor  Organization  (ILO)  focuses  on  necessary  basic-needs  of  technology-related  framework  for 
the  low  income  women/populations.  The  Technology  Programme  of  the  International  Labor  Organization  has  had  an 
antipoverty  thrust  to  improve  the  poor's  access  to  basic  goods  and  services  in  rural  areas.3  The  International 
Women's  Tribune  Centre  (IWTC)  works  in  partnership  with  the  International  Development  Research/Eastern  and 
Southern  Africa  Office  (IDRC/Nairobi).  This  organization  has  developed  a new  information  tool  that  offers  direct 


1 The  term  "digital  divide"  refers  to  the  gap  between  the  haves  and  have-nots  in  terms  of  the  access  to  the  computer 
and  the  Internet.  In  the  last  part  of  this  paper,  a boarder  definition  incorporated  various  factors  was  introduced. 

2 Refer  to  http : //www  .netschools.  net/wh ynetschool/c s hundred . htm 

3 Refer  to  http://www.idrc.ca/en/ev-28596-201-l-DO  TOPIC.html. 
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access  to  information  for  women  who  are  among  the  most  marginalized  in  development.  The  program  initiative  is  a 
CD-ROM  -Rural  Women  in  Africa:  Ideas  for  Earning  Money  for  rural  women  to  learn  via  a user  friendly  process. 

European  rural  women  are  not  a homogeneous  group.  They  have  different  roles  and  occupations,  and  work  on 
farms  and  in  family  businesses,  in  employment  and  in  community  activities.  But  they  have  been  affected  by  the 
globalization  processes  that  lead  to  the  restructuring  and  decline  of  the  agricultural  sector  and  the  growth  of  the 
service  sector  and  increased  emphasis  on  technology.4 

Patricia  Liths’s  (2002)  “Uganda:  ITCs,  empowerment  and  Women  in  rural  Uganda”,  investigated  African 
women’s  social,  technological  and  information  contexts.5  This  work  identified  barriers  to  women's  full  use  of  ICTs, 
and  then  developed  strategies  for  overcoming  those  barriers. 

In  Australia,  the  community  networking  and  interactive  communication  technology  (ICT)  projects  was  based 
on  feminist  or  social  justice  principles  usually  aiming  to  include  a broad  diversity  of  community  members.  Groups 
often  targeted  include  women,  indigenous  people,  people  of  non-English  speaking  backgrounds  or  with  low  incomes 
and  people  living  in  rural  and  remote  areas  (Harrison,  1998). 

Asia  and  South  Asia  present  showcases  in  providing  cutting-edge  expertise  to  drive  the  global  information 
technology  industry.  The  Asia-Pacific  region  shows  great  diversity  in  gender-related  indicators  and  differential  gains 
in  the  advancement  of  women  and  girls.  In  the  mid-1980s,  several  studies  looked  for  areas  where  improved 
technology  could  both  reduce  the  workload  of  and  generate  income  for  rural  women  in  South  Asia.6 

With  this  brief  glance  of  gender  and  technology  in  developed,  developing  and  underdeveloped  countries,  the 
following  section  focuses  on  the  conditions  in  the  U.S  and  reviews  research  related  to  IT  use  and  needs  from  a 
multistate  longitudinal  project  on  rural  low  income  mothers’  well  being. 

A Review  from  the  Domestic  Perspective 

In  America,  there  are  approximately  59-65  million  adults  living  in  rural  communities,  or  21%  of  the  U.S.  adult 
population.  Rural  Internet  penetration  has  remained  roughly  10  percentage  points  behind  the  national  average  - 67% 
of  urban  residents,  66%  of  suburban  residents,  and  52%  of  rural  residents  (PEW,  2003). 

As  expected,  among  rural  residents,  low  income  women’s  access  to  computers  in  general,  was  quite  limited 
They  tend  to  have  less  access  than  that  of  men,  urban,  and  suburban  women  to  information  technologies  (PEW, 
2003).  When  the  1996  Telecommunications  Act  was  passed.  Congress  was  explicit  that  telecom  service  in  rural 
areas  be  comparable  to  what  was  available  in  urban  areas,  including  access  to  advanced  telecommunications  and 
information  services.  At  the  current  stage  of  broadband  deployment,  rural  residents  do  not  have  equal  access  at 
reasonable  rates  despite  measures  by  the  FCC  to  insure  universal  access.7  Subsequently  the  National  Policy 
Administration  (NPA)  established  the  Digital  Economic  Opportunity  Committee  (DEOC)  to  examine  the 
information  technology  skills  gap.  The  committee’s  objective  is  to  turn  the  digital  divide  into  digital  economic 
opportunities  (NPA  Report,  2002).  Furthermore,  welfare  reform  policy  intends  to  support  the  low  income  mothers’ 
independence  by  providing  job  and  skill  training,  education  and  other  relevant  self-sufficient  strategies.  Based  on 
this  policy  implication,  we  were  curious  about  whether  the  computer/Internet  usage  played  a role  in  mothers’  daily 
lives  that  assisted  their  work  or  family  responsibilities. 

The  data  of  this  research  were  generated  from  the  Rural  Families  Speak  (RFS)8  - a multistate  longitudinal 
research  project  focusing  on  the  well-being  of  rural  low-income  families  in  the  context  of  welfare  reform.  The 
original  project  began  in  2000  across  17  states,  targeting  on  mothers  with  at  least  one  child  under  the  age  of  12 


4 Refer  to  http://europa.eu.int. 

5 Refer  to  http://www.wougnet.org/News/cdupdate.html. 

6 Refer  to  http://www.fao.org/docrep/005/ac788e/AC788E03.htm. 

7 Refer  to  http://www.ihellerstein.com/rural.html  (3  of  1 1). 

8 Rural  Families  Speak  Project  involves  two  project  identification  numbers  of  NC-223  and  now  NC-1011.  There 
have  been  researchers  in  seventeen  states  involved  at  different  periods  of  time.  Two  major  pieces  of  legislation  have 
driven  the  welfare  reform  discussions.  Public  Law  104-193,  The  Personal  Responsibility  and  Work  Opportunity 
Reconciliation  Act  of  1996  (PRWORA),  and  Public  Law  104-180  The  Agricultural,  Rural  Development,  Food  and 
Drug  Administration,  and  Related  Agencies  Appropriations  Act,  1997  (Food  Stamp  legislation).  The  legislation 
governing  Food  Stamps  appropriations  is  passed  each  year.  The  latest  one  is  Public  Law  106-387.  The 
reauthorization  of  the  PRWORA  continues  to  be  discussed.  There  are  other  pieces  of  legislation  that  influences  low- 
income  families,  but  the  team  is  focusing  on  these  pieces.  All  the  states  have  supported  the  data  collection  and 
preparation  of  data  set.  The  overall  purpose  of  the  research  is  to  assess  changes  in  the  well-being  and  functioning  of 
rural  families  in  the  context  of  policies  that  reform  welfare.  Refer  to 
http://www.cehd.umn.edu/fsos/centers/RuralFamiliesSpeak/proiectDescrip.asp 
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living  at  home  and  with  an  income  at  or  below  200%  of  the  federal  poverty  line.  Data  were  collected  from  413 
families  in  rural  counties  (with  population  centers  of  less  than  20,000).  This  was  not  a random  sampling  from  the 
fifty  states  but  was  a purposive  sampling.  Participants  were  interviewed  by  trained  researchers  for  approximately  2-3 
hours  in  a semi-structured  format  with  open-ended  questions  either  at  their  homes  or  in  another  mutually  agreed- 
upon  location.  Qualitative  and  quantitative  data  were  gathered  in  areas  such  as  community  and  housing 
characteristics,  child  care,  parenting,  financial  stability,  food  security,  employment,  transportation,  health,  mental 
health  and  education. 

Questions  regarding  low  income  mothers’  job  training,  educational  opportunities  and  family  life  revealed 
serendipitous  information  relating  to  computer  and  the  Internet  usages.  These  face  to  face  interview  results  provided 
an  opportunity  for  investigation  of  IT  use  in  the  rural  low  income  mothers’  lives.  Based  on  this  discovery,  we 
formulated  our  research  question  to  probe  into  low  income  mothers’  relationship  with  IT  - “ What  are  rural  low 
income  mothers’  perceptions,  experiences,  barriers  and  supports  relating  to  computer  and  Internet  usage ?” 

Basic  Demographic  Information 

Most  of  low  income  mothers  in  the  413  sample  are  non-Hispanic  white  (63.9  percent)  with  smaller 
proportions  of  Hispanic  (21.3  percent)  and  African-American  (8.7  percent)  and  other  ethnic  groups.  The  majority  of 
mothers  were  in  their  early  20s  to  mid-30s  (51.6  percent).  Almost  one  out  of  two  mothers  did  not  complete  high 
school  (45.5  percent)  or  had  some  college  training.  Close  to  half  were  either  married  (44.8  percent)  or  living  with  a 
partner  (14.8  percent).  The  average  family  income  is  $15,522.  One  third  (33.8  %)  of  them  had  an  Internet 
connection  which,  in  2001  when  this  data  was  gathered,  was  much  lower  than  the  national  average  (PEW,  2003). 

Method  and  Data  Analysis 

We  read  through  original  transcripts  several  times.9  A search  through  interview  texts  for  the  words  ‘computer” 
and  “Internet”  via  software  Maxqda2  revealed  165  cases  with  segments  containing  the  searched  expressions.  To 
examine  the  salient  themes  from  computer  usage  and  Internet  access,  we  tabulated  the  collected  segments  to  extract 
the  essences  that  potentially  contribute  to  a theme.  All  the  proper  segments  went  through  thematic  reduction  process 
and  qualitative  analysis  techniques  (Miles  & Huberman,1994;  Strauss  & Corbin,  2002). 

After  several  readings  of  the  extracted  texts  of  165  cases,  33  were  identified  with  meaningful  segments 
related  to  perceptions  and  experiences  with  information  technologies.  Based  on  the  above  criteria,  the  similar 
statements  of  participants’  expressions  were  grouped  into  categories  which  were  then  clustered  into  sub-codes  and 
codes. 

Through  back  and  forth  inductive,  deductive,  and  abductive  reasoning  processes,  similar  segments  were 
clustered  into  three  primary  themes:  general  usage,  barriers,  and  supports. 

The  primary  and  sub-codes  were  organized  and  explained  as  follows: 


9 During  2007-8, 1 had  an  opportunity  to  join  RFS  project  at  University  of  Minnesota-Twin  Cities  as  a research 
assistant  for  Dr.  Walker. 
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Figure  1:  Primary  Codes  and  Sub-codes  of  Rural  Low  Income  Mothers’  Using  IT 

General  usage 

The  general  usage  was  defined  as  the  common  usage  that  most  people  will/can  do.  The  sub-codes  are: 

a)  Employment,  including  working  with  computer;  job  training  with  computer 

b)  Seek  information,  including  job  hunting;  obtaining  education,  finance,  health  and  parenting  information. 

c)  Family  activities,  including  family  entertaining  and  edutaining,  such  as  parent  engaging  with  children  via 
computer  games;  parent  providing  educational  games  for  children;  and  monitoring  children’s  online  activities. 

d)  Business  activity,  including  online  business,  online  shopping,  downloading  free  merchandises,  and  applying 
public  assistance. 

e)  Communication,  such  as  online  dating,  online  chatting,  online  making  friends,  and  keeping  anonymity,  as  well  as 
personal  routine  activity  in  the  online  environment. 

Barriers 

A barrier  is  defined  as  the  interviewee’s  expression  of  frustrations,  obstacles  or  undesirable  conditions  that 
impede  access  to  computer  and  the  Internet.  The  sub-codes  are: 

a)  Financial  barriers:  hardware,  software,  and  accessories  expenditure;  the  maintenance  fee,  and 
connection  service  charges. 

b)  Accessibility  barriers,  geographical  obstacles,  such  as  lack  of  transportation  to  community  centers  or  social 
service  agencies;  physical  challenges  such  as  weights  illness  causing  the  immobility;  inadequate  electronic 
connection,  such  as  having  computer  yet  not  having  Internet 

connection  or  using  out-of-date  computer  and  accessories;  and  having  the  slow  dial  up  Internet  connection;  as 
well  as  personal  felony  records  prohibiting  the  access  opportunities. 

c)  Cognitive  barriers-  lack  of  IT  knowledge,  such  as  lack  of  the  basic  computer  skills;  being  a computer  illiterate; 
and  experiencing  frustrations  when  computer  breaking  down. 

d)  Time  barriers  - constraints,  such  as  juggling  between  motherhood,  house  chores,  jobs,  and  other  commitments. 
Supports 
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Social  supports  refers  to  the  functions  including  instrumental,  informational,  and  emotional  assistance  that  is 
provided  to  individuals  by  significant  others  including  family,  supervisors  and  friends  (Thoits,  1995),  and  by  social 
service  institutions  or  non-profit  organizations. 

a)  Supports  from  Individuals,  such  as  receiving  help  from  participant'  parents,  boyfriend,  partner;  close  relatives 
such  as  siblings;  and  friends  and  neighbors. 

b)  Supports  from  organizations,  including  obtaining  assistance  from  local  community  libraries  and  job  centers; 
social  programs  from  public  and  non-profit  organizations  providing  computer  training  certificate  programs  and 
access  to  software  and  hardware. 


Results 

General  usage 

In  the  general  usage  code,  the  most  frequent  category  of  use  was  family  activities.  Family  entertaining  and 
edutaining  with  children  were  the  major  responses  indicating  that  computers  were  used  in  the  homes  of  rural  poor 
mothers  by  their  children  or  for  mothers  to  interact  with  their  children.  Mothers  reported  that  the  most  available  and 
inexpensive  family  activities  that  most  children  enjoyed  related  to  working  on  computers,  doing  Internet  activities 
and  by  some,  playing  with  video  games  if  they  had  one.  Some  mothers  expressed  the  value  of  having  a computer 
available  to  their  children.  For  example, 

“Yes.  I'm  pretty  sure  he'll  go  to  some  kind  of  college  because  he  already  knows  how  to  use 
the  computer  and  he's  4 years  old.  I gave  him  my  old  one  to  play  with.  And  he  made  it  do 
something  that  I tried  for  months  to  figure  out  how  to  do.” 

Another  mother  expressed: 

“Johnny  likes  playing  computer,  and  Jenny  likes  to  watch,  so  we  have  educational  computer 
programs  on  the  computer,  that  he  can  play.  And  then  I sit  down  with  him  and  help  him 
with  some  of  them,  but  he's  a really  smart  kid.  So  he's  teaching  me  stuff  on  the 
computer  that  I didn't  know.” 

The  second  most  frequent  category  of  usage  in  the  general  usage  code  was  employment  relating  categories 
which  included  four  types  - working  with  computer,  job  training  with  computer,  job  hunting,  and  online  business. 
Most  mothers  describe  actual  use  of  computers  related  to  work;  work  was  planned  or  hoped  for  (e.g.,  working  with  a 
computer  being  an  ideal  job,  if  available;  looking  for  education  or  training  opportunities  to  enhance  computer  skills; 
and  having  desire  to  take  computer  courses  to  enhance  employability).  A mother  shared  her  successful  story  by 
taking  advantage  of  computer  training  opportunities,”!  changed  my  job.  I took  a couple  of  computer  classes  and  I 
moved  up  to  working  at  the  newspaper  in  the  city”  Finding  information  was  the  third  most  frequent  category  of  use 
and  communication  was  the  final  category  of  use. 

Barriers  to  IT 

The  major  barriers  the  participants  expressed  were  affordability,  knowledge  about  using  computer  and  time 

issues. 

The  cost  of  purchasing  and  maintaining  a computer,  or  paying  for  the  Internet  was  beyond  the  capacity  for 
many  families.  As  one  mother  plainly  said,  "No.  We  don't  have  a computer.  We  just  can't  afford  it”.  Others 
expressed  placing  payment  for  a computer  or  the  Internet  at  a low  priority,  after  other  utilities  and  household  bills 
were  paid.  Or  they  indicated  a combined  concern  for  cost  and  the  amount  of  time  that  their  children  were  spending 
on  the  computer.  Cost  was  also  a consideration  to  repair  a broken  computer.  As  one  mother  said,  “...cause  the  timer 
or  something-something  about  the  computer  was  off  and  instead  of  just  fixing  that  part,  they  just  have  to  do  the 
whole  thing  and  it  cost  a lot  of  money.” 

For  time  and  cognitive  barriers,  mothers  expressed  their  frustration  at  not  being  able  to  use  or  learn 
computers  because  of  other  demands  on  their  time.  For  example, 

.it's  my  intention  to  return  to  college  in  the  fall  of  2002.  I’ve  already  checked  into 
getting  him  (her  son)  into  Early  Head  Start. . . since  he  turns  three  before  September,  he 
would  automatically  be  eligible  for  the  program  for  next  September  for  all  day.  So  I could 
be  in  school  all  day  to  study  computer  science.” 

Supports  to  computer  and  IT  access 

To  overcome  the  lack  of  access  to  the  computer  and  Internet  as  well  as  limitations  imposed  by  the  high  cost 
of  various  service  charge  and  soft/hardware  problems,  rural  low  income  mothers  expressed  the  following  available 
resources  to  solve  their  own  problems. 
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Public  facilities . The  top  resources  are  the  local  community  centers  and  libraries  that  provided  access  to 
computer  facilities,  education  programs  and  job  training  opportunities.  Responding  to  a question  about  her  job,  one 
mother  mentioned  that  she  took  advantage  of  the  state  Adult  Education  program  of  50  hours  training  to  obtain  a pass 
for  the  computer,  she  said,  “ Cuz  that's  the  only  way  I would  spend  any  time  together  with  my  family. . ..That  way 
my  kids  can  teach  me  how  to  do  it.  Pretty  soon  my  4 year  old  could.  She  knows  more  about  computers  than  I do.” 
Another  mother  said,  “I  took  a couple  of  computer  classes  through  Community  Action  They  paid  for  them. . . I just 
decided  I would  like  to  do  that.” 

Workplace.  Workplace  also  facilitates  the  opportunities  to  access  the  Internet.  One  mother  occasionally  had  to 
deal  with  work  and  child  care  issue,  and  found  an  advantage  to  computer  access.  She  said, 

“So,  usually,  it's  been  a couple  of  times  that  I’ve  needed  childcare  and  1 just  take  her  to 
work  with  me. . .They  (the  employers)  don’t  say  anything  because  she's  really  good.  She 
doesn't  run  around,  she'll  stay  with  me.  The  last  time  I took,  I took  her,  I just 
took  her,  her  computer  games  in  the  computer  lab  (at  her  workplace.) 

Relatives  and  Neighbors.  Help  from  neighbors  to  share  the  usage  as  well  as  relatives’  assistance  are  extra 
resources  to  the  mothers.  One  luckier  mothers  appreciated  her  “possession”  by  saying,  “A  hand-me-down  from  my 
parents,  'cause  they  got  a new  one.  It's  a Compaq  ninety-five,  I think,  so  it's  pretty  old,  and  it's  pretty  slow,  but  it's 
better  than  nothing.  It  works.” 

Conclusion  and  Discussion 

General  speaking,  rural  low  income  mothers  revealed  the  uses  of  IT,  primarily  for  family  and  employment 
purposes,  which  encountering  barriers  to  access  and  relaying  on  family  and  friends,  community  and  employment 
resources  as  supports.  Cost  was  the  largest  barrier,  and  most  mothers  reported  not  having  the  Internet  in  their  homes. 
Based  on  mothers’  comments,  many  did  not  have  computers  at  all  due  to  the  cost  of  purchase  or  maintenance.10 
Using  computers  in  public  places  and  at  work  were  resources  for  mothers’  use.  Family  and  friends  also  were 
supports,  letting  mothers  use  their  computers,  or  giving  them  old  computers.  This  issue  resonated  Hargittai’s  (2002) 
the  double  levels  of  digital  divide:  the  first  level  of  computer  ownership  and  the  second-level  of  multiple  effects  on 
digital  divide,  which  extend  to  broadband  access;  machine  vintage;  connectivity;  online  skills;  autonomy  and 
freedom  of  access;  and  computer-use  support  (Hawkins,  Rudy,  & Nicolich,  2005). 

Horrigan  (2004)  pointed  out  that  solving  the  availability  problem  would  get  higher  IT  adoption  in  rural  areas. 
Broadband  technology  is  the  ideal  mechanism  to  narrow  the  divided  gap.  Practical  uses  of  technology  in  diverse 
learning  environments,  in  particular,  the  remote  rural  areas,  the  distant  killers  can  conquer  the  distance  barriers. 

Having  access  to  broadband  Internet  means  that  rural  low  income  mothers  obtain  new  kinds  of  support 
systems  to  provide  the  communication  mechanism  for  their  children,  schools,  teachers  and  other  stakeholders.  It 
opens  up  new  ways  of  exploring  resources  and  opportunities,  which  has  impact  on  children's  well  being.  Such 
learning  environment  includes  local  libraries,  community  centers,  and  K-12  schools  where  all  members  of  the 
community  have  access  to  meaningful  programs  and  providing  useful  lifelong  knowledge  and  skills. 

As  of  today,  we  know  Finland  is  the  first  country  to  announce  every  Fin  citizen  to  access  a 1 megabit-per- 
second  broadband  connection  in  next  year,  and  a 100  megabit-per-second  broadband  connection  at  the  end  of  2015. 
This  move  inspires  other  countries  to  start  perceiving  broadband  as  citizens’  inalienable  legal  right,  akin  to  part  of 
the  First  Amendment. 

Domestically,  many  non-profit  organizations,"  promote  a wide  range  of  free  broadband  access  and  neutrality 
issues.  They  have  been  organizing  grass-root  movements  to  raise  the  consciousness  and  to  persuade  policy  makers. 
For  example,  Socialfreenet  advocates  free  broadband  Internet  access  to  low -income  families  that  are  in  need  of  high- 
speed access  to  attain  financial,  educational  and  employment  goals.  We  suggest  that  US  federal  government  play  a 
more  active  role  to  shorten  the  digital  divide  in  broadband  access.  This  means  investing  and  renovating  IT 


10  At  the  end  of  2005,  only  24  percent  of  rural  Americans  had  broadband  access,  compared  with  39  percent  of  urban  and 
suburban  dweller,  according  to  Pew  Internet  & American  Fife  Project,  Rural  Broadband  Internet  Use,  memo,  February 
2006,  http://www.pewinternet.org/pdfs/PIP  Rural  Broadband.pdf. 

1 1 Such  as  Freepress  ( http://freepress.net/media  issues/interneO.  SaveThelnterent  (http://www.savetheinternet.com/).  Finding 
Dulcinea  (http://www.findingdulcinea.com/guides/Technology/Internet/Free-Use-Media.pg  05.html,)  , and  Socialfreenet 
(http://socalfreenet.org/mission). 
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infrastructures  at  local  public  organizations,  such  as  community  centers,  child  care  facilities,  transportation  agencies, 
job  centers,  libraries,  schools  and  churches  can  open  up  wider  opportunities  for  learning,  training,  and  guidance  to 
low  income  mothers. 
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Abstract 

This  is  an  introspective  account  of  the  author’s  lived  experience  in  facilitating  web-conference  (using  software 
Live  Meeting),  hybrid,  pure  online,  and  face  to  face  (f2f  ) with  web-enhanced  four  delivery  methods  for  4 courses 
of  Sociology,  Diversity,  Developmental  Psychology,  and  Contemporary  American  Society  in  the  spring  semester  of 
2009.  The  dynamics  of  coordinating  and  adjusting  structures  and  functions  of  each  learning  environment  among  the 
author,  students,  IT  department,  and  administration  policy  illustrates  the  challenges  of  both  non-instructional  and 
instructional  issues  situated  in  a highly  contextualized  setting. 

Introduction 

Technical  colleges  play  a key  role  to  bridge  PK-12  and  15  to  20  for  many  higher  educational 
institutions.  Though  most  two  year  technical  colleges,  comprehensive  community  colleges,  and  four  year  colleges 
tend  to  be  lumped  together  as  the  post-secondary  educational  system  or  “higher  educational”  institution,  they  are 
fundamentally  different  in  many  aspects.  Such  as  educational  missions,  climate,  diversity  of  student  and  faculty 
body,  specific  roles  of  instructor  and  staff,  funding,  infrastructure  and  the  overall  ecological  configuration,  just  to 
name  a few,  comprise  the  uniqueness  of  2-year  technical  colleges  that  stand  out  as  a special  and  controversial 
educational  entity.  These  are  the  main  reasons 

Due  to  the  historical  necessity,  2-year  colleges  (community  and  technical)  have  been  playing  unique  and 
multifaceted  roles  to  meet  the  socio-cultural  and  economic  needs  of  our  society.  Nevertheless,  the  issues  of 
scholarship  in  the  2-year  colleges  have  been  interpreted  with  different  values  and  emphasis  due  to  its  hybrid 
structure  and  functions.  Having  been  a grad  student  and  instructor  crossing  over  a research  oriented  educational 
institution,  a 4-year  polytechnic  university,  and  to  my  current  working  at  a 2-year  technical  college,  I have  been 
experiencing  the  role  of  “cognitive  migrant”  for  seven  years. 

Stokes  reflected  on  the  “Pasteur’s  Quadrant,”  suggesting  the  dual  focus  of  building  basic  theory  while 
simultaneously  improving  practice.  One  of  the  puzzling  issues  is  that  most  theories  and  practice  in  teaching  and 
learning  tends  to  focus  on  PK-12  or  at  the  university  level,  leaving  the  most  dynamic,  diverse,  demanding 
educational  setting  of  the  2-year  college  with  less  adequate  research  and  interests.  Lord  (1988)  once  pointed  out  that 
most  people  did  not  associate  2-year  colleges  with  the  terms  "scholarship"  and  "research."  One  reason  was  that  the 
college  mission  statements  rarely  included  "scholarship  and  research”  to  be  a necessary  part  of  “teaching 
excellence”.  Furthermore,  in  higher  education  most  people  define  scholarship  as  being  equivalent  to  publication. 
Other  academic  activities  outside  publishing  are  neither  noted  nor  appreciated.  This  led  to  some  stakeholders'  anti- 
elite style  of  publication  that  “neglected”  what  true  education  was  about.  The  conundrum  that  excellent  teachers 
will  not  be  competent  researchers  nor  excellent  researchers  rarely  are  good  teachers  reinforces  such  a stereotyping. 

The  question  is  that  if  pursuing  the  mutual  enhancement  between  theories  and  practice  is  an  important 
educational  goal,  then  balancing  research  and  teaching  is  a necessary  step  to  reach  that  aspiration.  If  this  is  the  case, 
then  only  focusing  on  teaching  without  the  input  from  the“formal  scholarly  activity  or  research"1  about  what  is 
really  happening  in  the  core  of  teaching  and  learning  is  similar  to  telling  a story  without  contents,  or  building  a 
house  on  the  loose  sands. 


1 Ernest  Boyer  (1990)  proposes  a broader  view  of  scholarship: 

1.  The  scholarship  of  discovery  2.  The  scholarship  of  integration  3.  The  scholarship  of  application 
4.  The  scholarship  of  teaching 
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Therefore,  this  case  study  intends  to  highlight  a lived  experience  to  bridge  research  and  practice  within  a 
unique  context  where  teaching  18  to  21  credit  loads  (an  average  of  7 classes)  per  semester  is  the  norm,  making 
scholarly  research  challenging.  Besides,  it  also  addressed  some  concerns,  such  as  the  emerging  participatory 
technologies  creating  a new  generation  of  learners  who  are  familiar  with  the  anytime,  anywhere,  and  anyhow,  as 
well  as  “any-what”  learning  style  and  demands,  in  addition  to  the  economic  uncertainty  that  reflects  in  educational 
budgetary  constraints  demanding  more  versatile  and  accountable  in  the  teaching  enterprise. 

To  attend  to  the  above  concerns,  my  research  question  is:  What  is  the  lived  experienced  of  a social  study 
instructor  facilitating  6 classes  via  four  delivery  formats  within  a semester  (from  January  to  May,  2009)? 

As  to  two  terms  in  this  paper,  the  “instructional  issue”  is  defined  as  any  concern  that  directly  relates  to  the 
basic  interaction  between  teachers  and  students.  The  ”non-instructional  issue”  is  defined  as  conditions  beyond 
virtual  or  physical  classrooms  in  the  school  setting. 

The  purpose  of  this  research  has  two  levels  of  significance.  At  the  individual  level,  this  paper  examines 
issues  from  the  instructional  perspective  to  obtain  first-hand  experience  to  highlight  2-year  college  scholarship 
issues.  It  also  surveyed  the  author’s  navigation  among  different  instructional  strategies  ( e.g.,  across  pedagogy, 
andragogy,  and  huetagogy)  between  constructivism  and  instructionism.  At  the  institutional  level,  the  author  also 
explored  issues  from  the  non-instructional  perspective  to  illustrate  the  dynamics  among  the  instructor,  students,  IT 
Department,  and  administration's  anticipated  learning  technologies  to  provide  accessibility  and  efficiency. 

Research  Background 

While  there  is  a considerable  literature  on  the  characteristics  of  successful  online  courses  and  on  how  to 
bring  good  pedagogy  into  the  online  learning  environment  (Conti,  1985b;  National  Center  for  Education  Statistics 
2001;  Allen  et  al.  2002;  Migliete  and  Strange  1998;  Post  et  al.  1998;  Parisot,  1997;  Lowes,  2008),  research  on  the 
multiple  delivery  formats  of  multiple  courses  on  teaching  and  learning  is  sorely  inadequate,  particularly  for  the  2- 
year  college  setting. 

In  online  learning  environment,  researchers  and  scholars  have  been  touting  the  potential  to  transition  higher 
education  from  a teacher-centered  orientation  to  the  constructivist  student-centered  approach  (Sherron  and  Boettcher 
1997;  Beaudoin  1990;  Dillon  and  Walsh  1992;  Chanchaem  2001;  Berge  1997;  Kearsley  2000).  But  other  researchers 
also  indicate  that  there  is  a discrepancy  between  the  promoted  learner-centered  style  and  what  is  happening  in  the 
online  learning  environment. 

Online  facilitation  is  different  from  the  face  to  face  instruction  in  terms  of  pedagogical  approaches  and 
learning  technologies  application  (Zhao,  2003).  However,  online  instructors  tend  to  carry  face  to  face  pedagogies 
into  the  online  environment  (Connolly,  Jones,  & Jones,  2007).  Sheely  (2006)  examined  factors  directed  to  answer 
“why  online  instructors  cannot  stop  lecture  online”.  Parisot  (1997)  stated  “little  has  been  done  to  understand  the 
changing  role  of  faculty  in  adapting  to  technology  and  the  changes  in  the  psychological  and  physical  environment 
promised  by  distance  learning”.  A decade  later,  Barrett  et  al.’s  research  echoes  such  observation,  “....each  new 
technological  advancement  engenders  great  expectations  regarding  its  impact  on  instruction.  Regardless  of  the 
changes  in  technology,  teaching  style  has  not  changed  and  remains  teacher-centered”  (Barrett,  Bower,  & Donovan, 
2007).  In  order  to  implement  quality  online  course,  Kochtanek  and  Hein  (2000)  contend  that  transformation  of  the 
instructor’s  role  from  instructor  to  facilitator  was  an  important  initial  step.  They  argued  that  a successful  student- 
centered  learning  environment  requires  that  the  instructor’ s role  to  change  from  a knowledge  transmitter  to  a 
knowledge  facilitator. 

Moving  debates  beyond  f2f  and  online,  within  the  limited  multiple  delivery  format  literatures,  Lowes 
(2005)  pointed  out  that  “trans-classroom  teachers”  experienced  very  different  cultures  and  social  practices  between 
two  delivery  environments:  online  and  f2f.  She  suggested  that  their  migratory  journey  to  and  from  the  online 
classroom  can  transform  teacher's  f2f  classroom  practice  in  subtle  and  important  ways.  Lurthermore,  Zirkle  et  al. 
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(1999)  pointed  out  the  transition  to  instruction  through  multiple  delivery  systems* 2  has  given  rise  to  several  issues 
and  concerns.  One  of  them  relating  to  this  paper  is  that  teaching  courses  through  multiple  delivery  formats  requires 
high  levels  of  expertise  on  the  part  of  faculty. 

As  aforementioned,  there  are  multiple  purposes  of  this  exploratory  study.  At  the  first  cycle,  this  paper 
concentrated  on  the  instructional  and  non-instructional  aspect  of  study.  It  also  focused  on  the  subjectivity  and  inter- 
subjectivity between  the  author  and  her  four  delivery  formats  in  examining  the  concepts  among  the  wide  spectrums 
of  pedagogies  between  instructionism  and  constructivism. 

Research  Method  and  Procedures 

To  gain  better  understanding  of  the  aforementioned  scholars’  research  and  suggestions,  the  author  situated 
herself  beyond  the  traditional  dichotomy  of  online  vs.  on-ground  by  adding  the  hybrid  and  the  currently  piloting 
web-conference  (using  Microsoft  Live  Meeting  to  replace  the  high  cost  ITV  and  increase  the  accessibility  to 
students)  into  the  repertoire.  With  the  above  understanding  in  mind,  through  a year’s  planning  and  preparation  via 
various  type  of  professional  development  programs,  the  author  obtained  the  opportunity  to  gather  four  major 
delivery  formats  started  from  last  semester  (beginning  on  January  12,  2009)-  online,  hybrid,  web-conference  via 
Live  Meeting  (similar  to  Adobe  Connect/Breeze),  and  traditional  face  to  face  with  web-enhanced  methods  for  her 
six  social  studies  courses:  three  sociology  classes-  one  f2f,  one  hybrid,  and  one  web-conference;  one  online 
Diversity  Study;  one  f2f  Developmental  Psychology;  and  one  f2f  Contemporary  American  Society.3 

The  method  applied  in  this  introspective  paper  was  an  auto-ethnographical  case  study.  The  “situated 
adventurous  teaching”  experiences  were  documented  in  the  daily  and  weekly  class  logs  for  each  delivery  format 
with  a modified  small  scale  of  design-based  research  component.  4 There  were  3 stages  of  construction  described  as 
follows: 

Constructing  the  Basic  Research  Conditions  and  Procedures 

1 . Prior  to  implementing  the  multiple  delivery  formats  : organizing  the  non-instructional  resources 

• Understanding  school’s  policy  and  concerns  regarding  delivery  formats 

• Participating  in  relevant  professional  and  curricular  development  projects  and  software  learning 
processes 

• Constructing  a sound  technical  supporting  network. 

2.  During  implementing  multiple  delivery  formats  : constructing  the  instructional  process  and  assess 

products 

• Constructing  collaborative  wikiblog  for  learners’  participation  and  reference. 

( e.g.,  http://cvtcsocialscience.wetpaint.com ) 

• Construct  experiential  teaching  Resources-Based  websites  for  personal  usage. 

(e.g.,  http://cvtcscholarship.wetpaint.com ) 

• Informing  incoming  students  and  gather  basic  demographic  information  and  technical  skills. 

• Discerning  student  prior  knowledge  on  subject  matters  and  technical  skills. 

• Documenting  teaching  and  learning  activities  in  class  logs  on  daily  and  weekly  basis  for  each 
delivery  format  (e.g.,  in  the  above  wikiblog  and  BlackBoard  master  copy  archive). 


In  Zirkle  et  al.’s  case,  students  have  the  option  of  taking  courses  through  one  of  four  delivery  formats:  traditional  on- 

campus  delivery,  IHETS  (Indiana  Higher  Education  Telecommunication  System)  satellite,  videotape,  and  the  Internet. 
Faculty  members  in  the  department  teaches  at  least  one  course  each  week  via  multiple  delivery.  The  faculty  teaches  the 
traditional  course  to  on-campus  students  in  a specially  designed  studio  while  distance  students  at  the  remote  satellite  sites 
participate  simultaneously.  After  each  class,  videotapes  of  the  session  are  mailed  to  those  students  who  have  requested 
that  mode  of  delivery.  The  instructor  then  accesses  the  course  Web  site  and  posts  lecture  notes,  presentation  slides, 

3 In  my  current  work  setting,  the  average  of  teaching  loads  per  semester  is  7 classes  or  21  credits.  The  national  average 
ranging  from  15  to  18  credits.  The  formal  research  and  publications  in  the  academic  community  is  not  required. 

4 My  original  design  research  method  intent  was  cancelled  after  the  first  3 weeks’  cycle  due  to  a frequent 
technical  difficulty  among  far  site  learners,  network  issues  and  software  affordances  during  the  pilot  period. 
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• Designing  and  implementing  cyclical  (3-5  weeks)  teaching  and  learning  conditions  and 
assessing  specific  pedagogical/andragogical  issues  in  each  cycle  from  the  instructor  designed 
feedback  system. 

• Redesigning  the  teaching  and  learning  environment  and  beginning  another  new  cycle. 

3.  At  the  end  of  the  whole  cycle  of  the  multiple  delivery  formats 

• Analyzing  the  class  logs  and  communication  documents  among  the  instructor  and  the  relevant 
administrators. 

Instrumentations 

Three  instruments  were  used  in  this  retrospective  study.  One  was  student  basic  information  survey  (age, 
gender,  GPA,  income,  employment  conditions,  level  of  technology  familiarity,  delivery  format  preference)  designed 
by  the  author  and  was  given  to  each  student  at  the  first  day  of  school.  The  other  one  was  “The  Principle  of  Adult 
Learning  Scale”  (PALS,  developed  by  Gary  Conti  in  1978)  to  be  my  self-report  to  discern  the  author's  teaching 
style.  The  last  instrument  was  a simple  self  report  on  performance  satisfaction5  measuring  pedagogical  fulfillment 
designed  by  the  author  for  this  paper. 


Results 

This  retrospective  auto-bio-ethnographical  analysis  went  through  inter-subjective  categorization  and 
reduction  processes  from  recorded  class  logs  and  communications  with  school  relevant  administrators.  I identified 
the  following  categories  emerged  from  the  data  analyses.  Three  levels  of  themes  emerged.6  Due  to  time  and  space 
constraints,  this  paper  focused  on  a theme  from  the  microscopic  aspect  of  “teaching  meeting  learning”  and  the 
relating  instructional  and  non-instructional  issues. 

The  Need  of  Differentiated  Instructional  Strategies 

Teaching  style  can  be  one  of  the  components  to  facilitate  a desirable  teaching  and  learning  environment. 
Dede  et  al.  (1999)  maintained  that  to  create  an  optimal  environment  for  learning,  instructors  need  to  consider  the 
confluence  of  not  only  technology  and  subject  matter  but  also  learner  characteristics  and  pedagogical  principles. 
These  are  the  ingredients  they  propounded  as  a true  synthetic  learning  environment  (SLE).  Instructors’ 


5 This  subjective  construct  was  based  on  f2f  practices  as  the  frame  of  reference.  There  are  4 items  with  Likert  scale  of  1 to  5 
designed  to  tailor  the  instructor’s  characteristics  to  assess  the  optimal  performance.  This  Self  Report  of  Scarification 
includes  the  following  items  briefly  described  as  follows: 

(1)  Technological  affordances  to  facilitate  pedagogical  fluidity:  Measure  each  format’s  capacity  to  support  a specific 
pedagogy  or  instructional  strategies.  For  example,  will  the  delivery  format  allow  the  instructor’s  spontaneity  in 
incorporating  learning  resources  to  happen  seamlessly?  Will  the  delivery  format  enhance  the  collaboration  both 
synchronously  and  asynchronously? 

(2)  The  need  for  the  instructor’s  course  management  time:  will  the  format  demand  extra  technical  preparation?  For 
example,  web-conference  Live  Meeting  demands  instructors  to  pre-load  learning  materials  in  a static  format  in  each 
meeting  session.  It  also  needs  extra  course  management  system,  such  as  BlackBoard  to  assist  relevant  learning  activities. 

(3)  Dynamic  interactive  capacity:  For  example,  when  I need  to  present  a video  from  Youtube  in  a timely  fashion,  will  the 
delivery  format  facilitate  that  possibility  without  latency?  In  our  current  Live  Meeting  software,  the  course  content  has  to  be 
preloaded  , and  once  preloaded,  the  content  becomes  static,  e.g.,  the  hyperlinks,  and  relating  interactive  functions  would  be 
lost.  No  document  camera  functionality  (like  the  old  Elmo)  was  available  to  demonstrate  serendipitous  information. 

(4)  Optimal  performance:  A delivery  format  enhances  the  strength  and  decrease  the  weakness  of  the  instructor’s 
characteristics  in  applying  a specific  instructional  strategy?  For  example,  personally  I love  podcast  for  my  online  class,  but 
the  accent  that  I carry  much  discourages  such  facilitation  after  a couple  of  attempts. 

6 These  three  themes  are  (1)  the  macro  level  relating  to  contextual,  or  school  ecological  issues,  such  as  school  climate, 
internal  and  external  policy  regarding  budget,  curriculum,  retention,  and  accreditation,  (2)  the  meso-level  regarding  the 
dynamics  between  classroom  (both  virtual  and  physical)  and  intra-institutional  practices,  such  as  student  recruitment  and 
orientation,  academic  service,  and  career  and  personal  counseling  , and  (3)  the  micro-level  of  what  really  happened  in  the 
physical  and  virtual  classrooms. 
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characteristics  can  also  add  up  one  more  dimension  into  their  repertoire.  The  instructor's  characteristics,  such  as 
teaching  philosophy,  personality,  and  academic  background  and  research  interests  also  play  an  important  role  in  the 
teaching  and  learning  environment.7 

In  the  SLE  model,  subject  matter  and  pedagogical  principles  are  controllable  and  manageable.  The 
technology  ingredient  largely  depends  on  the  interactions  among  instructors,  administrator  and  school  policies.  The 
most  challenging  factor  will  be  the  learner  characteristics.  The  open  admission  policy  of  2-year  colleges  accepts 
diverse  learners  based  on  first  come  first  serve  principle.  The  wide  spectrums  of  student  body  consists  of 
disproportionate  numbers  of  learners  who  work,  unemployed,  displaced  from  jobs,  raising  families,  or  arriving 
college  with  academically  unprepared  (AACC,  2006;  U.S.  Department  of  Education,  2007).  Without  various  types 
of  Federal,  State,  and  local  financial  supporting  systems,  majority  of  our  current  students  would  not  have 
opportunities  to  access  the  post-secondary  educational  opportunities. 

Via  “The  Principle  of  Adult  Learning  Scale”  (Conti,  1978)  self  report  test,  I identified  myself  as  a 
constructivist  type  of  learner  and  facilitator.8  On  the  student  side,  the  informal  student  basic  information  surveys  at 
the  beginning  of  the  semester  (age,  gender,  GPA,  income,  employment  conditions,  level  of  technology  familiarity, 
delivery  format  preference)  showed  the  consistency  with  the  profiles  of  2-year  college  learners  in  the  national 
statistics.9  Typically,  in  my  classes,  a majority  of  students  identified  themselves  as  working  class  or  (or  non-middle 
class,  from  the  annual  income  report),  as  well  as  the  first  generation  to  attend  college.  Learning  with  disabilities,  ex- 
offenders, high  school  dropouts,  displaced  workers,  returning  aspirants  for  further  knowledge  or  skill  training 
comprised  occupational,  remedial,  and  transfer  three  major  categories.  As  Cross  (1981)  pointed  out  by  the  time  the 
2-year  colleges  were  developed,  most  young  people  from  the  higher  socioeconomic  groups  and  most  of  the  high 
aptitude  aspirant  were  going  to  4-year  colleges.  And  the  majority  of  student  entering  open-door  2-year  colleges 
came  from  the  lower  half  of  the  high  school  class,  academically  and  socioeconomically.  This  depiction  of  the 
general  student  profiles  also  provided  an  epitome  of  the  continuous  debate  over  the  issue  of  hybridization  of  2-year 
colleges  in  terms  of  being  a comprehensive  institution  with  university  transferring  preparation,  or  narrowing  its  roles 
by  offering  vocational  training,  adult  and  remedial  education.  No  matter  which  path  the  2-year  would  go,  the 
diverse,  under-served  and  disadvantaged  student  population  will  be  continuously  growing. 

Facing  the  increasing  diverse  learners,  though  not  all  agreeing  upon  a simplified  formula,  Ertmer  and 
Newby  (1993)  suggested  to  combine  different  theoretical  perspectives  into  a simplified  formula: 

“....Employ  the  behaviorist  perspective  in  situations  in  which  learners  have  lower  levels  of  task  knowledge 
and  for  learning  goals  requiring  lower  cognitive  processing;  use  the  cognitivist  perspective  for  middle 
levels  of  task  knowledge  and  cognitive  processing;  and  consider  the  constructivist  perspective  for 
situations  in  which  learners  have  a higher  level  of  prior  knowledge  and  are  working  on  higher  level  tasks, 
such  as  complex  problem  solving  in  ill-structured  domains”  (pp.  68-69). 

Between  a constructivist  instructor  and  structured  guidance  needed  learners  posed  many  challenges  in 
delivery  methods.  Gagne  ( 1995)  pointed  out; 

“. . ..there  are,  after  all,  some  useful  human  activities  that  are  acquired  without  instruction,  and  others  that 
result  from  self-instruction.  But  most  practical  and  purposeful  activities,  such  as  the  pursuits  involves  in 
vocational  and  technical  training,  are  learned  in  settings  that  employ  instruction”  ( p.  17). 


7 For  example,  in  terms  of  self-efficacy,  being  a cross-cultural  instructor,  leaving  the  cultural  comfort  zone  and 
moving  back  and  forth  within  and  between  various  teaching  and  learning  subcultures  is  part  of  daily  routines. 

My  multi-disciplinary  academic  background  (5  majors  and  4 minors) , prior  working  knowledge  and  experience  in 
military  and  government  as  well  as  the  current  Learning  Technologies  major  also  play  a supportive  role  for  this  adventure. 
These  advantages  facilitate  the  author  to  take  on  multiple  delivery  formats  for  multiple  courses  as  part  of  her  teaching  and 
learning  endeavors. 

s In  last  10  years’  student  evaluations  and  comments,  I got  the  pattern  of  recognition  as  a passionate  and  helpful  instructor 
mirroring  the  constructivist  type  of  description. 

9 

As  of  2009,  the  national  profiles  show  there  are  1,177  2-year  colleges  with  1 1.7  million  enrollments,  which  comprise  44% 
of  the  total  undergrads.  Among  them.  58%  are  female,  the  average  age  is  29.  Above  21  years  of  age  is  53%,  and  minority 
counts  for  38  % of  total  enrollments  (with  13%  of  African  Americans  and  16%  of  Hispanic  is  16%,  7%  of  Asian/Pacific  7 
%,1  and  1%  of  the  Native  Americans).  40%  of  students  enrolled  as  full  time,  and  60%  are  part-time. 
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Fortunately,  the  proliferation  of  research  has  provided  results  across  the  spectrums  between  radical 
constructivism  and  minimal  guidance  instructions.  These  solutions  provide  alternative  instructional  strategies  (e.g., 
derived  from  pedagogy,  andragogy  and  heutagagy)  for  different  audiences  and  learning  goals. 

Differentiated  instructional  strategies  involve  providing  students  with  different  avenues  to  acquiring 
content;  to  processing,  constructing,  or  making  sense  of  ideas;  and  to  developing  teaching  products  so  that  all 
students  within  a classroom  can  learn  effectively,  regardless  of  differences  in  ability  (Tomlinson,  2001).  This  type  of 
synthetic  or  differentiated  method  is  a solution  in  my  daily  virtual  and  physical  classrooms. 

From  the  instructional  or  pedagogical  perspective,  the  instructor  needs  to  be  equipped  with  a wide  range  of 
instructional  strategies  falling  between  the  highly  sUuctured  direct  guidance  on  one  site  to  the  ill  structured 
facilitation  at  the  other.  Inside  the  toolkit,  based  on  learner  characteristics  and  delivery  format’s  technological 
affordances,  adjusting  appropriate  pedagogies,  strategies  and  migrating  among  different  teaching  and  learning 
environments  is  a key  factor  to  be  successfully  engaging  the  diverse  learners  in  the  different  delivery  formats. 

The  following  themes  described  such  interface  between  the  pedagogies  and  delivery  formats  from  the  4 
delivery  formats  of  my  spring  semester  2009. 

The  Symbiotic  Relations  between  Pedagogy  and  Delivery  Formats 

(1)  Face  to  Face  Delivery  Format 

Among  the  4 delivery  formats  during  spring  2009,  f2f  was  the  most  manageable,  effective  and  efficient 
method  compared  to  other  3 methods.  I heavily  relied  on  non-verbal  communications  and  spontaneity  in  my 
teaching.  10  “Well  structured”  learning  activities  or  hands-on  projects  tends  to  fit  majority  learners’  needs. 
Approximate  15%  of  students  were  self-motivated  with  stronger  subject  matter  and  relevant  prior  knowledge,  who 
fit  into  various  types  of  pedagogies.  Among  this  able  group,  3-5  % showed  a similar  type  of  the  expertise  reversal 
effect. 


From  the  non-instructional  perspective,  this  format  was  the  least  demanding  IT  support  except  minor 
classroom  multimedia  station  problems  which  usually  students  and  I could  fix  without  sending  service  requests.  To 
avoid  sending  tickets  became  a reflex  starting  3 years  ago.  For  school's  scrupulously  budgeting  issues,  the 
outsourcing  IT  jobs  to  private  company  made  requesting  timely  technical  service  difficult.  F2f  class  teaching  also 
earns  the  lowest  teaching  loads  - 14.4%  per  class  with  27  students  as  the  cap.  11  In  this  delivery  environment,  I had 
highest  Self  Report  on  Satisfaction  score  with  4.5  points  on  the  5 point  scale.  The  student  evaluation  was  above 
average  (3. 3/4.0).  So,  among  the  4 formats,  f2f  was  my  most  favorite  delivery  method. 

(2)  Online  Delivery  Format 

Though  the  design  principle  provided  social,  cognitive  and  content  presence  for  the  online  facilitation,  it 
mainly  lost  my  signature  of  spontaneity  and  dynamics  as  an  expressive  instructor  who  heavily  relies  on 
dramaturgies  and  body  languages!  Though  missing  the  idiosyncratic  features,  I supplemented  with  timely  feedback 
to  compensate  the  missing  ingredients.  Interestingly  speaking,  I earned  the  highest  evaluation  among  the  4 methods 
(3. 5/4.0)  which  meant  that  without  physical  presence,  I did  better  job  than  that  of  my  dynamic  f2f  class  performance! 
One  speculations  of  mine  was  that  the  online  learners  were  exempt  from  my  third  language  accent  and  way  of 
expressions  that  I had  “imposed”  on  my  f2f  classes. 

The  other  factor  was  that  the  prospective  online  learners  were  encouraged  to  take  READI  test  to  obtain 
online  learners’  capability  profile  in  terms  of  reading  comprehension,  technical  skills,  and  self  motivational 


10  For  I am  not  a native  speaker-  English  is  my  third  language. 

1 1 Between  90  to  105  % is  counted  as  full  time  teaching.  Currently  administration  and  union  negotiations  resulted  in  the 
following  load  counting:  each  f2f  is  14.5%,  online  15%,  hybrid  15.6%,  web-conference  18%,  and  high-school  ITV  is 
21%. 
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characteristics.  This  step  functioned  as  a filter  to  discourage  learners  who  were  not  ready  for  online  classes. 
Generally  speaking,  online  learners  possess  higher  reading  and  writing  ability  than  that  of  students  in  other  delivery 
formats. 


From  the  non-instructional  perspective,  BlackBoard  is  our  course  management  system  (LMS).  Though  the 
constant  updated  versions  are  available  to  enhance  interactive  possibilities,  due  to  the  cost,  the  basic  version  did 
fulfill  major  teaching  and  learning  goals.  Recent  studies  show  the  limitations  of  proprietary  LMS  or  CMS  systems 
do  not  match  up  the  emerging  ubiquitous  and  serendipitous  participatory  learning.  However,  since  majority  of  our 
students  were  not  from  middle-income  households,  the  high-end  devices  and  high-brow  cognitive  capacity  were  not 
part  of  the  key  discourse.  In  this  delivery  environment,  my  own  Self  Report  on  Satisfaction  score  was  at  4 points  on 
the  5 point  scale-  my  second  favorite  delivery  format. 

(3)  Hybrid  Delivery  Format 

Hybrid,  according  to  the  available  literatures,  has  been  touted  as  the  best  of  the  two  world  method.  As  an 
educator,  I enjoyed  hybrid  facilitation.  This  method  allows  a wide  spectrum  of  pedagogical  maneuvering  in  both 
settings.  Nevertheless,  it  also  posed  challenging  to  incorporate  two  types  of  pedagogy  and  two  systems  of 
interactions  and  learning  assessments  which  demand  much  more  work  in  course  management  and  complicating  the 
mental  migration  between  the  two  worlds  within  a delivery  format. 

From  the  non-instructional  perspective,  again,  due  to  budget  issues,  eliminating  pure  online  courses  to  be 
substituted  with  the  less  expensive  hybrid  offering  has  been  a policy.  The  negotiation  of  class-size  and  work-loads 
has  been  a hot  button  issue  between  administrators  and  the  teachers’  union. 

In  this  delivery  environment,  my  Self  Report  on  Satisfaction  score  was  at  2.5  points  on  the  5 point  scale, 
due  to  the  complexity  of  migrating  between  two  worlds  and  students'  preferring  the  online  part  of  engagement  and 
less  interest  in  the  resident  requirement  (at  least  50%  of  the  seat  time  for  every  hybrid  course)  for  variety  of  reasons. 

(4)  Web-conference  Delivery  Format 

Last  semester  (spring  2009)  was  the  second  phase  of  the  pilot  program,  and  was  my  first  attempt  to  try  out 
this  method.  So  far,  this  format  consumed  most  of  my  teaching  time.  Among  6 classes  of  4 courses,  40%  of  the 
preparation  and  teaching  loads  was  spent  on  this  method  due  to  the  limited  software  affordance  to  support  certain 
type  of  pedagogies.  The  need  to  facilitate  student  learning  with  intense  emails  and  BlackBoard,  as  one  facilitator  put 
it,  “ It  was  equivalent  to  teaching  two  courses  in  one  format  - one  online  and  the  other  one  was  audio  live 
conference.”  Data  collected  from  the  first  phase  pilot  showed  majority  of  students  (144  students  in  9 classes) 
expressed  the  negative  evaluation  on  this  method.  A total  of  8 Instructors  participating  in  the  pilot  program 
presented  50-  50  positive  and  negative  evaluation  on  this  delivery  format  (CVTC,  2009). 

Personally,  Live  Meeting  web  conference  presented  tremendous  challenge  in  intended  pedagogies.  Via 
web-conference,  due  to  the  network  technical  difficulty,  the  webcam  installation  decreased  the  audio  quality  of  all  3 
far  sites.  Thus,  the  multifunctional  web-conference  was  regressed  into  an  “audio  synchronous  conferencing”.  Audio 
synchronous  communication  exposed  my  unique  characteristics  as  a non-native  speaker  in  this  environment. 
Compared  to  my  previous  3 semesters  of  facilitating  ITV  (interactive  TV  for  multiple  learning  sites),  the  real-time 
virtual  visual  communications  compensated  the  proximity  inadequacy  . The  multiple  remote  site  learners  were  able 
to  see  and  interact  with  far-site  learners  and  conducted  collaborative  projects  in  the  synchronous  fashion.  Thus, 
comparing  between  ITV  and  Live  Meeting  conferencing,  I prefer  the  former  due  to  the  visual  interactions  between 
far  site  learners  and  me. 

From  the  non-instructional  perspective,  the  network  supporting  system  and  students’  technical  skills  were 
the  essential  for  Live  Meeting  web  conference  to  be  functioning  well.  Both  learners  and  facilitators  need  to  be 
familiar  with  Live  Meeting  software  and  BlackBoard  LMS  systems.  Unfortunately,  each  system  has  its  own 
limitations.  The  limited  technological  affordances  confine  certain  pedagogical  possibilities. 

The  major  reason  for  piloting  this  method,  mainly  was  a top  down  decision  making  due  to  the  budgetary 
concern  and  intended  to  provide  an  alternative  access  for  students  who  would  not  be  able  to  take  other  formats.  It 
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resulted  in  replacing  the  much  more  expensive  yet  more  effective  ITV  delivery  method.  In  this  delivery  environment, 
my  Self  Report  on  Satisfaction  score  was  the  lowest  at  1 point  on  the  5 point  scale. 

Conclusion  and  Discussion 

This  reflective  paper  resulted  from  a multi-levels  of  involvement  with  school  system.  It  was  an 
idiosyncratic  monologue  within  the  author  and  her  delivery  formats.  Prior  to  taking  off  the  multiple  delivery 
formats,  the  original  idea  has  been  evolving  from  previous  years  conscious  planning  and  preparation.  The  author 
used  her  situated  and  experiential  teaching  conditions  to  address  two  major  concerns  relating  to  technological 
affordances  and  pedagogies  in  one  of  the  emerged  themes.  These  two  major  concerns  are:  (1)  the  paucity  of 
research  and  interest  in  the  fields  of  2-year  colleges  relating  to  learning  technologies,  in  particular,  the  multiple 
delivery  formats,  (2)  the  circumstances  between  a differentiated  knowledge  transmitter  and  knowledge  facilitator 
associated  with  instructional  and  non-instructional  factors  that  affected  teaching  and  learning  engagement. 

Overall,  what  lesson  did  I learn  from  this  pre-meditated  adventure?  I concluded  that  being  a pan- 
classroom12 facilitator  enhances  the  following  conditions:  13 

• Being  an  adventurous  instructor,  flexibly  situated  oneself  in  an  diverse  teaching  environment. 

• Being  an  organic  cognitive  migrant,  fluidly  migrating  among  multiple  teaching  milieus. 

• Being  a 4A  Self-organizer,  accustomed  to  bridging  the  anytime,  anywhere,  anyhow,  and  any-what 

(informal  learning)  learning  conditions. 

• Being  a serendipitous  facilitator,  comfortably  being  a ubiquitous  4A  life-long  learner. 

• Meeting  school  mission  statements  to  facilitate  diverse  students’  access  and  intended  success. 

As  to  other  positive  and  non-positive  experiences  being  a multiple  delivery  faculty  at  a 2-year  college  setting,  I 
concluded  as  follows: 

1.  The  positive  effects  are: 

(1)  At  the  psychological  level: 

• Enhancing  self  efficacy. 

• Being  aware  of  technological  affordances  in  each  environment. 

• Being  adaptive  to  the  pedagogical  adjustments  within  and  between  different  delivery  formats. 

(2)  At  the  pedagogical  level: 

• Being  responsive  to  diverse  students’  characteristics  and  learning  needs  in  different  formats. 

• Being  able  in  evaluating  hardware  and  software  capacities  and  limitations. 

• Being  at  ease  to  integrate  contents,  pedagogies  and  technology. 

(3)  At  the  socio-cultural  level: 

• Creating  opportunities  to  network  with  IT  department  and  relevant  administrators. 

• Being  a better  problem  solver. 

• Being  a part  of  change  agent  team. 

2.  The  non-positive  effects  are: 

(1)  Community  of  practice:  it  is  not  a popular  journey  because  not  many  faculty  members  choose  to 

deliver  multiple  methods.  Thus  the  community  of  practice  is  inadequate  for  further  professional 

development  via  the  wisdom  of  the  crowd. 

(2)  Idiosyncratic  features:  It  exposes  my  idiosyncrasy  in  certain  delivery  format,  such  as  the  audio  web- 

conference  which  does  not  manifest  my  pedagogical  strength. 


Lowes  coined  the  trans-classroom  teachers  as  instructors  who  facilitate  and  migrate  between  f2f  and  online.  In  this  paper, 
I used  "pan-classroom  teachers”  as  instructors  who  migrate  between  more  than  two  environments. 

13 

It  might  not  be  the  case  in  other  faculty’s  condition.  For  example,  I am  the  only  faculty  member  in  the  department  having 
the  Learning  Technologies  as  one  of  my  majors.  For  some  faculty,  facilitating  we-  conference  can  be  an  unfathomable 
nightmare.  The  multiple  delivery  format  entails  knowledge  management  skill,  some  hardware  knowledge  and  software 
knowledge,  adjusting  the  cognitive  migration  among  different  delivery  formats.  The  simplest  way  to  start  multiple  delivery 
is  to  begin  with  hybrid,  then  move  toward  other  combinations. 


202 


(3)  Teaching  loads  and  scholarship  climate:  the  emphasis  on  teaching  with  an  average  of  21  credits  or  7 
class  teaching  loads  leads  to  tremendous  difficulties  to  conduct  research.  So  how  to  nurture  a 
scholarly  environment  to  promote  the  Pasture’s  Quadrant  is  an  imminent  yet  complicated  task. 

Further  research  is  needed  to  address  the  macro-  and  meso-level  issues  beyond  merely  the  interface 
between  learning  technology  affordances  and  pedagogies.  Furthermore,  the  economic  uncertainty  and  other  non- 
instructional  factors  could  impose  constraints  on  the  instructional  sphere,  thus,  enhancing  the  variety  of  pedagogical 
practices  and  adopting  technological  affordances  will  be  a solution.  Additionally,  the  emerging  participatory 
learning  entails  educators  to  be  versatile  and  adaptive  in  integrating  emerging  technologies  with  pedagogies  and 
content  knowledge. 

Empirical  study  at  this  micro-level  regarding  multiple  delivery  formats  can  be  extended  to  the 
measurements  of  effectiveness  and  efficiency.  Forming  a larger  community  of  practice  from  faculty  who  facilitate 
multiple  delivery  formats  can  facilitate  that  goal.  From  this  aspect,  it  is  also  urgent  to  call  for  a systemic  change  in 
2-year  college  scholarly  engagements  to  redefine  the  educator’s  roles  in  bridging  theories  and  practice  and  in 
integrating  educational  technologies  to  facilitate  diverse  students'  learning  needs. 
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Abstract:  With  the  development  of  the  use  of  ICT  in  education  in  China  entering  a new  phase,  the 
original  analysis  framework  has  difficulty  in  further  guiding  the  further  development.  This  paper  brings 
forward  a new  analysis  and  evaluation  framework  for  the  use  of  ICT  in  education  based  on  CIPO  Model, 
which  consists  of  four  levels:  Context  Analysis  and  Evaluation,  Input  Analysis  and  Evaluation,  Process 
Analysis  and  Evaluation,  as  well  as  Output  Analysis  and  Evaluation.  The  model  analyzes  the  use  of  ICT 
in  education  from  the  macro,  meso  and  micro  perspectives,  in  hope  of  bringing  new  inspirations  to 
educators,  such  as  policy  makers,  researchers,  facilitators,  practitioners,  and  IT  manufacturers,  etc. 

Keywords:  CIPO  Model,  ICT  in  Education,  Context  Analysis,  Input  Analysis,  Process  Analysis, 
Output  Analysis 


1 Introduction 

Along  with  using  of  ICT  in  education  in  China  transforming  from  the  phase  of  "large-scale  deployment  period" 
into  that  of  "using  skill  upgrading  period",  all  levels  of  education  authorities  are  actively  preparing  the  future 
development  plan  for  ICT  in  education.  The  premise  of  formulating  a scientific  and  reasonable  plan  is  a 
comprehensive  analysis  and  evaluation  of  the  use  of  ICT  in  education  in  China.  This  paper  will  give  an  in-depth 
analysis  for  the  four  aspects  of  ICT  in  education  based  on  CIPO  model. 

The  CIPP  model  put  forward  by  Stufflebeam  (2003)  not  only  is  widely  applied  in  the  educational  field,  but  also 
provides  a more  comprehensive  guidance  for  the  assessments  of  various  project  managements,  especially  for  those 
projects  that  are  ready  to  be  carried  out  for  a long  time  and  hope  to  acquire  sustainable  improvement.  CIPP  model  is 
also  known  as  "decision-oriented  evaluation  model",  consisting  of  four  parts,  which  are  Context  evaluation.  Input 
evaluation.  Process  evaluation  and  Product  Evaluation,  each  of  which  provides  different  information  for 
decision-makers.  The  greatest  value  of  CIPP  model  is  that  it  gives  a detailed  study  on  the  evaluation  subjects  from 
the  background  of  its  occurrence  and  development  to  the  affective  factors,  development  process  and  its  results, 
which  has  great  significance  and  value  for  the  overall  review  and  guidance  of  the  project. 

2 Evaluation  Framework  For  the  Use  of  ICT  in  Education:  CIPO  Model 

This  research  sets  up  CIPO  model  of  use  of  ICT  in  education  (Zhang  Jinbao,  2008)  with  the  reference  for  the 
two  core  ideas  of  CIPP  model,  one  of  which  is  introducing  the  objectives  into  evaluation  field  and  evaluating  current 
objectives  themselves  before  the  process  of  context  evaluation,  while  the  other  is  emphasizing  on  the  evaluation 
throughout  the  whole  process.  Based  on  it,  the  research  further  develops  it  into  an  analysis  and  evaluation 
framework  of  ICT  in  education.  See  Figure  1.  And: 
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• Context  Analysis 

• Problem  Diagnoses 

• Needs  Analysis 

• Objective  Determination 


• Compare  Similar  Projects 

• Set  up  Hardware,  Software  and 
Potential  ware; 

•Analysis  Budget 


Dimensionl: 
Context 
Analysis  and 
Evaluatio 


Dimension  4: 
Output 
Analysis  and 
Evaluation 


1 
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• Promote  the  Reform 
Level  of  Teaching  and  Learning' 

• Promote  the  Reform  Level  of 
Schools 

• Promote  the  Reform  Level  of 
Education 


.earning 

ivel  of 


Dimension  2: 
Input 

Analysis  and 
Evaluation 


Dimension  3: 
Process 
Analysis  and 
Evaluation 


• Pay  Attention  to  the  Subject 

• Analyze  the  Applicative 
Content  Evaluate  Promotion 
Mode 

• Deepen  ICT  in  Education; 


Figure  1 Analysis  and  Evaluation  Framework  of  ICT  in  education  Based  on  CIPO  Model 

(1)  Context  analysis  and  evaluation.  A combination  of  factors  that  affect  the  objectives,  necessity  and 
feasibility  of  ICT  in  education  include  the  aspects  of  education,  culture,  economy,  technology,  institution  and  so  on. 
In  this  part,  the  "information  technology  industry"  in  the  original  "six  elements  of  informationization"  should  be 
considered  as  the  element  of  this  level; 

(2)  Input  analysis  and  evaluation.  That  is  the  part  that  a variety  of  ICT  in  education  projects  directly  invest  in, 
generally  including  hardware  input,  software  input,  service  input  (including  training)  and  supporting  input.  In  this 
part,  the  "infrastructure",  "resource  development",  "professionals",  and  "policies,  regulations  and  standards"  in  the 
original  "six  elements  of  ICT  in  education"  should  be  considered  as  the  elements  of  this  level; 

(3)  Process  analysis  and  evaluation.  That  includes  the  issues  such  as  the  subjects  of  the  application  (initiator, 
facilitator,  transforming  agents  and  policy  makers),  application  contents  (concepts  and  methods,  tools  and  systems), 
promotion  mode  (decision-making,  adoption  and  dissemination),  and  deepening  the  application  and  integration  of 
informationization; 

(4)  Output  analysis  and  evaluation.  That  focuses  on  the  analysis  of  promoting  the  reform  at  three  levels, 
which  are  the  reform  of  teaching  and  learning,  that  of  schools  and  that  of  educational  system. 

The  introduction  of  ICT  in  Education  evaluation  and  analysis  framework  based  on  CIPO  model  makes  the 
original  over-flat  analysis  framework  have  a clearer  hierarchy  and  the  status  and  role  of  each  element  in  the  whole 
ICT  in  Education  becomes  more  accountable.  In  particular,  the  process  of  ICT  in  Education  changes  from  the 
original  " utilization  of  information  resources  and  application  of  information  technology"  to  four  operational 
elements  including  "human  beings",  "content",  "promotion",  and  "implementation",  which  has  more  significant 
meaning  of  guidance  for  researchers  and  practitioners. 
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Next,  we  will  analyze  each  of  the  four  aspects  involved  in  the  model  and  discuss  the  questions  that  need  paying 
attention  to. 

3 Context  Analysis  and  Evaluation  of  ICT  in  education 

For  the  education-oriented  informationization,  the  international  circumstances,  such  as  globalization  and 
diversity,  and  the  domestic  circumstances,  such  as  education,  science  and  technology,  culture,  society  and  economy 
are  all  external  factors,  which  cannot  be  ignored,  in  the  development  of  ICT  in  Education.  Globalization  is  a wave  of 
social  rationalization  and  integration  which  takes  the  economy  as  the  guide,  the  values  as  the  core,  the  politics  as  the 
supplement  and  the  general  culture  as  the  main  body;  while  informationization  is  the  process  transforming  from  the 
society  dominated  by  the  material  production  to  that  dominated  by  information  industry.  One  of  the  main  reasons  for 
the  countries  all  over  the  world  to  compete  in  investing  in  ICT  in  Education  is  the  inevitable  product  in  the  trend  of 
globalization  and  informationization,  especially  that  the  rapid  development  of  information  industry  demands  for 
opening  up  much  broader  market  space  for  itself. 


The  increased  educational  costs 


Request  for  reducing  the  cost 
of  education  and  call  for  the 
education  for  all. 


A wave  of  social 
rationalization  and 
integration  which  takes 
the  economy  as  the 
guide,  the  values  as  the 
core,  the  politics  as  the 
supplement  and  the 
general  culture  as  the 
main  body 


The  technology  is  updated  too  fast 
for  the  teachers  to  keep  up  with  its 
development. 

Education  cannot  afford  the  cost  of 
the  latest  science  and  technology. 
The  training  of  the  teacher 
resources  cannot  keep  up  with  the 
development  of  technology.  Schools 
cannot  keep  up  with  new 
technology. 


Behaviorism  vs. 
Constructivism. 

Subject  teaching  vs. 
multiple  subject  teaching 


Information  literacy  has  become  a 
basic  literacy  of  human  beings  in  the 
21s1  century. 

The  development  of  modern 
communications  has  seriously 
interfered  into  personal  privacy.  More 
and  more  people  do  not  trust 
technology. 

The  degree  of  dependency  on  the 
Internet  has  increased. 

The  phenomenon  of  information 
plagiarizing  has  come  up  in  great 
amount. 


The  process  transforming  from  the 
society  dominated  by  the  material 
production  to  that  dominated  by 
information  industry 


Students  from  poor  families  are 
difficult  to  acquire  technology. 
Men  are  dominant  in  the  field  of 
technology. 

Cost  of  the  technology  that  is 
available  for  students  with  special 
needs  is  very  high. 


Figure  2 Context  of  ICT  in  Education 

Since  the  link  between  the  objectives  of  economic  development  and  social  progress  and  the  development  of 
education,  especially  that  of  ICT  in  Education  was  established,  strong  political  overtones  have  been  added  to  the 
development  of  ICT  in  Education.  The  16th  National  Party  Congress  has  set  up  a strategic  goal  of  building  up  a 
well-off  society  in  an  all-round  way  at  a higher  level  in  favor  of  the  population  of  over  1 billion  by  2020.  Thus,  the 
Chinese  government  is  actively  adapting  itself  to  the  need  of  international  competition  in  the  circumstances  of 
globalization,  and  also  fully  understands  that  informationization  is  the  general  trend  of  the  development  of  the 
present  world  as  well  as  the  important  power  that  promotes  the  economic  and  social  reform.  Education  has  been 
endowed  with  a key  role  in  responding  to  the  challenges  of  globalization  and  informationization  by  the  Chinese 
government.  Through  the  process  of  the  16th  National  Party  Congress,  the  3rd  and  4th  plenary  sessions  of  the  16th 
National  Party  Congress  and  the  National  Conference  on  Human  Resources,  the  strategy  of  rejuvenating  the  country 
through  science  and  technology  and  education  and  that  of  reinvigorating  the  country  through  human  resource 
development  have  been  gradually  clarified. 

By  examining  the  relationships  among  various  contradictions  within  education,  we  can  also  see  that  people 
have  great  expectations  for  ICT  in  Education.  From  a macro  perspective,  the  education  in  China  faces  the  following 
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four  critical  challenges,  which  are  the  lack  of  investment  in  education  in  general,  extremely  unbalanced  development 
of  education,  unreasonable  personnel  structure  and  mode  for  talent  training,  and  incomprehensive  ideas  of 
educational  institutional  reform  (Zhang  Jing,  2005).  And  those  aspects  are  influencing  the  ideas  and  approaches  of 
the  educational  reform  and  development  in  China.  As  information  technology  has  obvious  advantages,  it  makes  ICT 
in  Education  be  regarded  as  an  important  force  to  facilitate  or  accelerate  the  educational  reform  and  development  in 
China.  As  early  as  in  1998,  an  official  of  the  Ministry  of  Education  of  China  has  put  forward  the  assertion  that 
modern  educational  technology  is  the  commanding  height  and  breakthrough  of  education  reform  and  development, 
which  has  been  further  verified  to  be  an  ME  proposition  by  the  academia  (He  Kekang,  1999). 

Above  all,  the  development  of  ICT  in  Education  has  a strong  historical  and  social  background.  It  is  the 
requirement  of  the  nation  in  realizing  economic  development,  social  development,  educational  development  and 
technology  development  and  so  on.  In  such  a broad  context,  the  implementation  of  ICT  in  Education  requires  us  to 
be  aware  of  the  related  requirements  generated  by  all  aspects,  to  comprehensively  review  them  and  to  design 
systematically  so  as  to  formulate  the  strategic  objectives  for  the  development  of  ICT  in  Education. 


Distance  Education  has 
more  advantages  in  costs 
and  profits 


It  will  become  one  of 
the  trends  of 
educational 
informationization  to 
expand  the  scape  of 
education  by  distance 
education  and  other 
technologies. 


The  application  of  the  latest 
technology  in  education 
comparatively  falls  behind. 
There  are  many  “out-of- 
date”  equipment  and 
resources  in  schools.  Most 
teachers  have  not  received 
appropriate  training  on 
technology.  Schools  cannot 
acquire  the  great  power 
brought  by  the  latest 
technology. 


Computers  have  become  a 
necessity  in  learning. 

Educators  have  the  obligations  to 
supervise  students  for  surfing  the 
Internet  and  to  construct  an  e- 
learning  environment  in  favor  of 
students’  development. 

Students  from  poor 
families  must  have 
equal  opportunity  to 
acquire  the  technology. 
Women  should  be 
offered  more 
opportunities  of  using 
computers  in 
education.  Provide  as 
many  opportunities  to 
use  the  technology  for 
the  disadvantaged 
group  as  possible. 


Change  the  simple  guidance  idea  of 
using  computers  to  adopting  more 
utilization  of  new  technology  in  the 
guidance  of  constructivism.  Insist  on 
using  the  software  good  for  single  skill 
and  use  the  technology  that  supports 
comprehensive  activities  of  multiple 
subjects  as  often  as  possible. 


Figure  3 Analysis  and  Evaluation  of  the  Input  of  Task  for  ICT  in  Education 
A research  report  (OECD,  2005)  shows  that  the  infrastructure  of  ICT  in  education  is  becoming  more  and  more 
popularized  both  in  developed  and  developing  countries.  In  the  "six  elements  framework  of  ICT  in  Education  " that 
we  are  familiar  with,  four  of  them  are  the  directly  operational  objects  to  promote  ICT  in  Education,  which  are  the 
"infrastructural  construction",  "information  resources  construction",  "training  for  informationization  talents"  and 
"policies,  regulations  and  standards  of  informationization",  corresponding  to  the  three  sections,  which  are 
"hardware",  "software"  and  "potential  ware",  in  the  investment  of  ICT  in  education 

"Infrastructural  construction"  refers  to  a variety  of  equipment,  such  as  computer  equipment,  networking 
equipment,  mobile  terminals,  multimedia  equipment,  satellite  equipment  and  their  construction  and  maintenance.  It 
aims  at  equip  current  educational  institutions  through  the  use  of  ICT  equipment  to  promote  educational  innovation 
and  development  so  as  to  enhance  the  quality  of  education.  On  the  premise  of  the  constraint  of  funding  for  education, 
how  to  provide  more  advanced  and  faster  computing  devices  for  education,  how  to  provide  more  secure  and  stable 
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network  equipment  with  wider  bandwidth  and  faster  response  speed  for  education,  how  to  provide  more  handy 
multimedia  equipment  and  mobile  terminals  with  higher  definition  for  education  which  can  better  satisfy  the  needs 
of  education  are  the  problems  to  be  solved  in  this  aspect.  As  the  specialty  of  the  education  field,  profit  and  funding 
are  difficult  to  guarantee,  so  ICT  in  Education  is  required  to  solve  the  problems,  such  as  "effective  input", 
"on-demand  design",  "strengthening  the  maintenance  and  management,"  "improving  efficiency",  and  "enhancing 
profit"  and  so  on. 

"Information  resources  construction"  can  be  divided  into  resources  related  to  supporting  teachers’  instruction, 
those  related  to  supporting  students'  learning  and  those  related  to  supporting  educational  management.  Digitization 
of  information  resources  is  an  effective  way  of  sharing  educational  resources  of  high  quality,  and  also  a necessary 
precondition  to  carry  out  the  integration  of  information  technology  and  curriculum  in  education.  This  aspect  has 
received  the  amount  of  investment  from  the  nation  only  second  to  the  aspect  of  "infrastructural  construction".  Over 
recent  years,  it  has  got  a large  amount  of  educational  resources  and  established  a number  of  focused  educational 
resources  databases.  As  the  construction  of  educational  resources  is  not  simply  digitalized,  but  concerns  the  needs  of 
users,  so  the  top  priorities  of  current  construction  of  information  resources  are  how  to  get  more  investment  to 
develop  the  themed  educational  resources  with  more  specific  focuses,  how  to  strengthen  the  mechanism  of  resource 
sharing  and  opening  up,  including  standards  for  resource  construction  in  order  to  deliver  more  free  resources  to  the 
units  in  need  of  them  and  how  to  build  up  a more  intelligent  resource  searching  and  classification  system. 

"Training  for  informationization  talents"  includes  not  only  IT  teachers  and  other  technical  supporting  staff,  but 
also  the  informational  literacy  and  informationization  skills  of  subject  teachers  and  school  administrators,  which  is 
of  great  significance  to  ensure  the  smooth  development  of  ICT  in  Education  and  also  is  the  important  service  in  the 
investment  of  ICT  in  Education.  Training  for  informationization  talents  has  to  make  great  efforts  to  train  IT  teachers, 
subject  teachers  and  students  as  well  as  to  strengthen  the  building  of  multidisciplinary  faculties. 

"Policies,  regulations  and  standards  of  informationization"  includes  the  investment  planning  and  financial 
policies  of  ICT  in  Education,  the  incentive  policies  for  encouraging  the  application  of  information  technology,  and 
various  series  of  specifications  and  requirements  for  hardware,  software  and  services,  etc.,  which  is  an  important 
guarantee  of  ICT  in  Education.  However,  the  problems  in  this  aspect  are  the  coordination  among  national  policies 
and  the  differences  of  systematic  educational  reform  and  the  reform  itself,  which  cannot  be  solved  by  a single 
reform  approach. 

"To  build  for  use"  is  an  incontestable  truth.  However,  so  many  challenges  that  present  ICT  in  Education  faces 
cause  people’s  doubt  about  its  construction.  One  important  aspect  is  how  to  measure  the  cost  and  profit  of  ICT  in 
Education.  According  to  the  research  report  by  World  Bank,  currently  there  still  lacks  an  effective  measurement  for 
the  costs  of  the  construction  of  ICT  in  Education.  Therefore,  it  is  very  necessary  to  focus  on  the  research  into  the 
costs  of  ICT  in  Education. 

Here  are  some  research  fields  worth  focusing: 

(1)  The  contents  of  total  costs  of  ICT  in  Education,  such  as  opportunity  costs,  equipment  costs,  maintenance 

costs,  ownership  costs,  calculation  method,  etc. 

(2)  Correlation  study  on  the  input  and  output  of  the  investment  in  education  in  different  regions,  different 
phases  and  different  educational  methods.  For  example,  which  kind  of  ICT  in  Education  should  be  currently  carried 
out  in  western  China  to  produce  maximum  profit? 

(3)  The  problem  of  costs  for  achieving  the  objectives  with  different  technical  plans  and  different  forms  of 
media.  For  example,  cost  differences  in  Internet  access,  cost  differences  between  resources  development  and 
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delivery  and  maintenance,  etc. 

(4)  Research  into  the  cost  of  specific  forms  of  ICT  in  Education.  For  example,  the  launching  costs  in  distance 
education,  the  average  cost  of  learners,  the  average  cost  of  graduate  students,  as  well  as  the  problem  of  cost 
transferring  and  the  enormous  cost  in  the  using  and  maintaining  the  donated  computers. 

(5)  The  impact  of  financing  mechanisms  of  ICT  in  Education  on  cost  saving,  such  as  the  methods  of 
government  investment,  public-private  partnerships,  local  credit,  personal  credit,  etc. 

4 Analysis  and  Evaluation  of  the  Promotion  Process  of  the  use  of  ICT  in  Education 

Although  the  infrastructure  is  gradually  popularized,  the  effect  of  application  has  not  reached  the  required  level. 
A very  important  reason  is  the  problem  in  promoting  the  application  of  ICT  in  Education.  In  the  past,  the  analysis  of 
the  application  of  ICT  in  Education  mainly  focuses  on  the  input  elements  of  ICT  in  Education,  such  as  system 
design,  equipment  performances  and  maintenance,  resource  development,  the  quality  of  the  users  and  so  on,  but 
seldom  focuses  on  the  process  of  application,  which  prevents  the  proper  promotion  of  the  application  of  ICT  in 
Education. 

Based  on  the  theory  of  Diffusion  of  Innovation  by  Rogers  (1995)  and  Concern  Based  Adoption  Model  by  Hall 
et  al  (2004),  we  divide  the  analysis  and  evaluation  of  the  promotion  process  of  the  application  of  ICT  in  Education 
into  four  aspects: 

(1)  The  main  body  of  ICT  in  Education,  including  all  kinds  of  educational  policy  makers  and  administrators, 
initiators,  facilitators  and  the  ultimate  users  of  the  application  of  ICT  in  Education,  As  the  differences  of  roles, 
different  stages  in  career  (eg,  pre-service,  in-service,  and  post-service),  individual  factors  (such  as  life  stages, 
families,  hobbies,  personal  characteristics,  life  crisis,  critical  events,  etc.),  social  factors  (such  as  rules  and 
regulations,  management  methods,  the  public  trusts,  social  expectations,  experts'  opinions,  the  units  one  works  for, 
etc.),  differences  of  innovation  and  degrees  of  risks  (such  as  pioneers,  early  adopters,  early  majority,  late  majority, 
those  who  are  left  behind),  differences  of  the  adoption  and  decision-making  process  (such  as  autonomous 
decision-making,  collegial  decision-making,  authorized  decision-making),  the  communication  and  exchanges  among 
the  main  bodies  of  ICT  in  Education  has  become  the  key  to  the  application  of  ICT  in  Education. 

(2)  The  content  of  the  application  of  ICT  in  Education,  including  the  application  of  information-oriented 
concepts  of  education  and  instruction  (also  known  as  "innovative  concept"),  methods  of  instructional  design  and 
instructional  strategies  (also  known  as  "innovative  technology"),  information-based  tools  and  information-based 
systems  (also  known  as  "innovative  product")  and  so  on  in  education.  Because  they  are  different  in  comparative 
advantages,  compatibility,  complexity,  the  features  for  observation  and  experimentation,  there  will  be  great 
differences  in  the  concrete  promotion  process,  which  has  to  draw  the  attention  of  all  practitioners  in  the  field  of  ICT 
in  Education. 
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(3)  Promotion  mode  of  ICT  in  Education.  It  directly  affects  the  speed  and  effects  of  the  proliferation  of  the 
content  of  the  application  of  ICT  in  Education.  The  effects  of  dissemination  and  proliferation  are  affected  by  media 
channels  (such  as  mass  media,  interpersonal  communication,  administrative  orders,  etc.),  dissemination  mode  (such 
as  point-to-point  mode,  vertical  dissemination  mode,  horizontal  dissemination  mode,  etc.),  degrees  of  users’ 
engagement  in  the  innovation  and  development  (such  as  function-oriented  development,  user-oriented  development, 
demand-oriented  development,  reform-oriented  development,  etc.). 

(4)  Deepening  information-oriented  education.  It  is  the  key  to  realize  sustainable  development  of  ICT  in 
Education  and  ultimately  realize  the  goal  of  ICT  in  Education.  Users  experience  three  phases  of  "focusing  on 
oneself",  "focusing  on  work"  and  "focusing  on  the  effects  in  cognition,  experience  eight  phases  of  "unused", 
"orientation",  "preparation",  "rote  implementation",  "routinization",  "specialization",  "integration",  and  "update"  in 
behavior.  Adjusting  the  promotional  activities  and  process  according  to  the  users’  feedback  and  providing  necessary 
support  for  them  are  indispensable  work  in  promoting  ICT  in  Education  to  a further  development. 

It  should  be  pointed  out  that  the  concrete  application  of  present  ICT  in  Education  is  often  constrained  by  four 
aspects,  which  are  the  effectiveness  of  application,  the  specificity  of  application,  the  routinization  of  application  and 
the  conditions  of  application.  Those  problems  have  been  affecting  the  whole  process  of  the  application  of 
informationization,  and  they  are  also  the  four  aspects  of  promoting  the  application  of  ICT  in  Education. 

5 Analysis  and  Evaluation  of  the  Output  of  ICT  in  Education 

The  analysis  of  the  output  of  ICT  in  Education  reflects  people's  attention  to  the  effects  of  ICT  in  Education.  On 
the  one  hand,  people  are  eager  to  know  the  status  quo  of  its  development;  on  the  other  hand,  it  is  the  evaluation  of 
the  degree  to  which  the  objectives  have  been  achieved.  Because  people's  expectations  and  purposes  for  ICT  in 
Education  are  different,  the  focuses  on  the  analysis  of  the  output  of  ICT  in  Education  also  vary.  According  to  the 
report  by  World  Bank,  there  are  not  widely  accepted  criteria  for  evaluation  at  present.  For  example,  the  criteria  of 
ISTE  for  students,  teachers,  administrators,  technical  staff,  "Performance  Indicator  System  of  ICT  in  Education" 
developed  by  UNESCO  (2003)  and  so  on.  In  addition,  different  countries  have  obvious  differences  in  the  evaluation 
indicators  (shown  as  Table  1). 


Table  1 Evaluation  Index  Levels  and  Indicator  Types  of  the  Effects  of  ICT  in  Education  in  Some  Countries 


Countries 

Levels* 

Qualitative  Evaluation 
Indicator 

Quantitative  Evaluation 
Indicator 

USA,  Canada,  Europe  (Denmark,  Sweden,  Finland,  Norway  and  UK), 
Australia,  South  Korea,  South  Africa 

3 

V 

V 

New  Zealand 

2 

V 

V 

Japan,  India,  Malaysia,  Thailand,  Uzbekistan,  Commonwealth  of 
Independent  States  countries  and  Baltic  States  countries 

2 

V 

Indonesia,  the  Philippines,  Vietnam,  Slovenia 

1 

* Level  1:  only  includes  input  indicator;  Level  2:  includes  input  and  process  indicators;  Level  3: 


includes  input,  process  and  output  indicators. 

This  paper  argues  that,  although  there  are  certain  differences  among  the  evaluation  system  developed  by 
different  countries  or  agencies,  we  cannot  arbitrarily  determine  that  the  higher  the  level  of  evaluation  is,  the  more 
comprehensive  it  will  be.  The  starting  point  of  selecting  or  developing  their  own  evaluation  indicators  should  be 
based  on  their  own  objectives,  that  is  the  purpose  of  the  evaluation.  According  to  the  author's  understanding,  the 
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analysis  of  the  output  of  ICT  in  Education  can  be  divided  into  three  levels,  which  are  "promoting  the  reform  of 
teaching  and  learning",  "promoting  the  reform  of  school"  and  "promoting  the  reform  of  education”. 

5.1  The  Level  of  “Promoting  the  Reform  of  Teaching  and  Learning” 

The  ultimate  objective  of  ICT  in  Education  should  lie  in  promoting  the  development  of  students  and  teachers’ 
profession.  The  Minister  of  Education,  Mr.  Zhou  Ji  said,  "We  should  insist  on  fostering  people-oriented  and 
student-centered  education  and  insist  on  talent-oriented  and  teacher-centered  operation  of  schools.  The  core  of  this 
dialectical  relationship  is  two  points.  What  is  the  development  of  our  education  for?  For  people.  How  should  we 
develop  education?  By  people.  "(Zhou  Ji,  2005)  The  application  of  information  technology  in  education  should 
firstly  point  to  the  development  of  people  in  education.  Reflected  in  the  output  of  ICT  in  Education,  the  teaching 
reform  in  the  context  of  ICT  in  Education  reflects  changes  in  three  aspects,  which  are  the  change  in  the  roles  of 
teachers  and  students,  the  change  in  the  process  and  methods  and  the  change  in  the  evaluation  process.  Therefore, 
the  analysis  of  this  level  should  focus  on  the  facilitating  effects  of  the  changes  in  the  above  three  aspects. 

However,  the  research  into  the  analysis  of  the  output  of  the  level  of  "promoting  the  reform  of  teaching  and 
learning"  shows  that  the  popular  four  approaches  of  the  application  of  ICT  in  American  school  education  are  still 
computer  teaching,  lesson  preparation,  instructional  inquiry,  as  well  as  word  processing  and  presentation.  The 
successful  mode  of  the  integration  of  ICT  in  or  out  of  school  instruction  is  still  unclear.  In  other  words,  further 
efforts  are  needed  for  promoting  ICT  in  Education  in  the  reform  process  of  teaching  and  learning  process  (World 
Bank,  2007). 

The  report  by  OECD  points  out  that  currently  ICT  is  far  from  being  the  core  of  the  learning  process,  although  it 
is  believed  that  ICT  will  have  a positive  impact  on  learning  performances;  and  there  is  still  a lack  of  evidence  for  a 
positive  impact  of  ICT  in  Education  on  students’  performances.  However,  some  researches  show  that  if  ICT  in 
Education  is  related  to  education,  it  will  be  much  easier  to  have  a positive  impact  on  students’  performances;  and 
from  the  perspective  of  motivation,  the  application  of  ICT  in  education  effectively  encourages  teachers  and  students. 
And  some  other  researches  finds  that  the  opportunity  of  obtaining  ICT  will  influence  users’  confidence  and  the 
utilization  of  ICT  can  enhance  learners’  autonomy.  Those  are  part  of  the  issues  to  be  focused  on  when  analyzing  the 
output  of  ICT  in  Education  on  this  level  (OECD,  2005). 

5.2  The  Level  of  “Promoting  the  Reform  of  Schools” 

The  reform  of  ICT  in  Education  involves  the  reforms  of  school  management,  instructional  working  habits,  and 
infrastructure  and  so  on.  However,  any  reform  is  subject  to  the  school's  cultural  traditions,  management  system, 
teachers’  working  conditions,  working  habits  and  working  services.  Thus,  a comprehensive  reform  is  difficult  to 
break  through  only  by  the  school’s  own  culture  and  system  evolution.  Unless  someone  with  strong  achievements  in 
technology,  theory  and  instruction,  etc.  strongly  push  it,  together  with  the  institutions  and  organizations  with  strong 
educational  backgrounds,  backgrounds  of  educational  technology  theory  as  well  as  technical  problem-solving 
capacity  that  break  the  cultural  inertia  by  external  attacks  in  order  to  promote  the  reform  of  schools. 

On  the  level  of  the  reform  of  schools,  data  collection  and  decision  support  provided  by  informationization  help 
to  improve  the  scientific  and  planning  school  management,  to  promote  the  implementation  of  school-based 
management,  to  realize  the  “off  center-oriented”  in  the  process  of  school  management,  and  to  realize  openness  and 
transparence  of  school  administrative  affairs.  Therefore,  those  are  important  aspects  of  informationization  in 
promoting  the  reform  of  schools. 

Both  OECD  countries  and  those  with  low  level  of  development  have  some  successful  experience  about  the 
utilization  of  ICT  on  the  level  of  promoting  the  reform  of  schools.  Some  of  the  successful  experience  has  been  well 
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documented,  such  as  the  series  of  reports  on  "Education  for  Tomorrow"  by  OECD.  However,  on  the  whole, 
successful  experience  or  failure  lessons  of  ICT  in  Education  are  rarely  widely  disseminated,  or  there  are  no  easily 
acceptable  approaches  presented  for  decision  makers;  nor  have  they  been  clearly  examined  in  the  educational 
context  related  to  the  Millennium  Development  Goals.  Therefore,  there  is  still  a lot  of  work  to  be  done  by 
informationization  on  the  level  of  the  reform  of  school  management  system. 

5.3  The  Level  of  “Promoting  the  Reform  of  Education” 

On  the  level  of  promoting  the  reform  of  educational  system,  ICT  in  Education  will  finally  make  the  realization 
of  the  “Education  for  All”  possible  and  help  to  reduce  the  uneven  distribution  of  educational  resources  among 
different  regions  to  achieve  fair  requirement  for  education.  In  addition,  the  enhancement  of  the  degree  of 
informationization  will  also  contribute  to  the  diversification  of  educational  objectives  and  evaluation. 

The  report  by  World  Bank  points  out  that  the  utilization  of  ICT  in  improving  the  efficiency  of  educational 
organizations  and  educational  system,  including  the  application  of  it  in  anti-corruption  work  in  the  departments  of 
education  is  currently  the  very  period  of  high  rate  of  return  for  developing  countries  (World  Bank,  2007). 

Research  report  points  out  that  on  the  level  of  the  reform  of  educational  system  level,  different  countries  have 
different  government  departments  responsible  for  the  work  of  ICT  in  Education  and  successful  policies  for  ICT  in 
Education  need  to  take  the  main  bodies  of  multiple  interests  into  account.  The  report  also  shows  that 
informationization  does  play  a good  role  in  the  field  of  special  education,  such  as  in  helping  the  handicapped,  etc., 
which  will  contribute  to  the  realization  of  educational  equity,  but  the  informationization  itself  has  the  possibility  of 
expanding  unfairness  to  form  the  so-called  digital  divide.  In  the  areas  with  a higher  marketing  degree, 
informationization  helps  to  the  dissemination  and  sharing  of  educational  contents. 

6 Conclusions 

Various  factors  push  forward  the  necessity  of  making  great  efforts  in  developing  ICT  in  Education,  and 
promoting  the  development  of  ICT  in  Education  definitely  requires  us  to  thoroughly  understand  the  inherent 
regulations  by  implication.  As  the  ICT  in  Education  in  China  has  entered  a new  stage  of  development,  it  is  the  goal 
that  researches  relentlessly  pursuit  to  look  at  ICT  in  Education  from  a higher  and  more  comprehensive  perspective 
so  as  to  have  a positive  impact  on  future  theoretical  research  and  practical  activities.  In  this  paper,  the  author 
attempts  to  develop  ICT  in  Education  analysis  framework  based  on  CIPO  model  from  the  six-element  model  of  ICT 
in  Education.  As  the  first  attempt,  the  analysis  of  mutual  relations  and  internal  elements  is  slightly  stiff  with  many 
deficiencies.  However,  the  author  firmly  believes  that  the  analysis  framework  with  the  general  background  of 
informationization  development  as  the  important  starting  point,  "hardware",  "software"  and  "potential  ware"  as  the 
input  elements,  four  aspects  of  promoting  "the  application  process"  as  the  process  and  the  realization  of  “the  reform 
on  three  levels”  as  the  output  reflected  by  the  analysis  and  evaluation  framework  presented  in  this  paper,  will  bring 
new  inspiration  to  educators  of  ICT  in  Education  such  as  policy  makers,  researchers,  facilitators,  practitioners,  IT 
manufacturers,  etc.  from  the  macro,  meso  and  micro  perspectives,  and  will  provide  important  guide  for  promoting 
educational  modernization  by  ICT  in  Education  in  the  new  five-year  development  stage  in  China  as  well  as  realizing 
the  leapfrog  development  of  education. 
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“We  are  hitting  home  runs!” 

Investigating  Early  Impacts  of  Online  Professional  Development  Courses 
on  School- wide  Systemic  Improvement  in  New  York  City  Schools 

Dr.  Jae-Eun  Joo 
WIDE  World 

Harvard  Graduate  School  of  Education 


I.  Introduction 


WIDE  World  (WIDE)  at  the  Harvard  Graduate  School  of  Education  offers  research-based  online  and  onsite 
professional  development  (PD)  programs  to  help  educators  achieve  systemic  and  sustainable  educational  changes  by 
improving  student  learning,  teaching  and  leadership.  WIDE’s  12-week  online  courses  aim  to  support  teachers  and 
leaders  to  understand  and  enact  research  proven  and  practical  strategies  through  interactive  and  coach-supported 
learning  activities.  Often  accompanied  by  onsite  workshops,  WIDE  online  courses  also  encourage  teachers,  teacher- 
leaders,  and  school  leaders  to  apply  ongoing  reflections  and  collaborative  teamwork  to  their  everyday  practices  to 
establish  schools  as  coherent,  professional,  and  systemic  learning  communities.  The  current  topics  of  WIDE  courses 
include  teaching  for  understanding,  differentiating  instruction,  engaging  multiple  intelligences,  integrating 
technology  into  curricula,  data-driven  educational  improvement,  and  school  leadership. 

In  August  2007,  WIDE  received  a grant  from  the  Harvard  Club  of  New  York  (HCNY)  Foundation  to  support 
professional  development  for  New  York  City  educators.  WIDE  World  doubled  the  impact  of  this  generous  funding 
by  requiring  participating  schools  to  match  the  grant  funds,  thereby  enabling  130  teachers  and  leaders  from  six  New 
York  City  (NYC)  elementary  and  middle  schools  to  take  advantage  of  WIDE  World’s  offerings  in  the  school  year  of 
2007-08  (Year  1).  Moreover,  this  grant  also  allowed  the  WIDE’s  Research  and  Evaluation  Department  to  carry  out 
an  in-depth  case  study  to  document  and  analyze  early  impacts  of  WIDE  on  educational  improvement  in  key  areas  of 
learning,  teaching,  and  leadership  in  these  schools.1  Thanks  to  the  successful  outcomes  of  this  first  cohort,  the 
HCNY  Foundation  approved  of  a second-year  grant,  which  was  also  matched  by  participating  schools,  to  extend 
WIDE’s  collaboration  with  NYC  schools  into  2008-09  (Year  2). 


From  September  2007  to  May  2009,  a total  number  of  184  teachers  and  leaders  in  six  of  NYC’s  public  schools  have 
taken  the  following  eight  WIDE  online  courses.2 


- DI 

- DW 

- uni 

- MI 

- TfUl 

- TfU2 

- TSNT1 

- TSNT2 


Differentiating  Instruction:  Strategies  for  Effective  Classroom  Practice 
Getting  Started  With  Data  Wise 
Leading  for  Understanding  1 

Using  Multiple  Intelligences  as  a Tool  to  Help  Students  Learn 
Teaching  for  Understanding  1 : Focus  on  Student  Understanding 
Teaching  for  Understanding  2:  Understanding  in  Practice 
Teaching  to  Standards  with  New  Technologies  1 
Teaching  to  Standards  with  New  Technologies  2 


1 Preliminary  Progress  Report  to  the  Harvard  Club  of  New  York  Foundation  (September  2007  - April  2008) 

2 Refer  to  the  WIDE  World  website  at  http://wideworld.gse.harvard.edu  for  more  detailed  information  on  these  courses. 
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Three  schools  out  of  the  six  Year  1 schools  continued  their  participation  in  Year  2.  All  located  in  Bronx,  the  schools 
and  their  principals  are: 

Connected  Neighborhood  Elementary  School  (CNE):  Principal  Jones 
Every  Child  Elementary  School  (ECE):  Principal  Miller 
Technology  and  the  Arts  Middle  School  (TAM):  Principal  Ellis3 

One  of  WIDE’s  main  research  goals  is  to  understand  both  the  process  and  the  specific  impacts  of  WIDE  courses  on 
instructional  practices,  student  outcomes,  school  leadership,  and  on  broader  system-wide  (e.g.,  school  or  district) 
improvement.  By  providing  high-quality,  research-based  professional  development  courses  using  networked 
technologies,  WIDE  aims  to  support  teachers  and  leaders  in  educational  systems  to  achieve  more  coherent, 
transformative,  and  long-lasting  improvement  on  a wide  scale. 

In  recent  years.  United  States  schools  have  been  preoccupied  with  meeting  accountability  requirements  mandated  by 
legislation  such  as  No  Child  Left  Behind  Act  (NCLB)  for  all  public  schools  and  students.4  These  requirements 
include  annual  testing  for  all  students  in  grades  3-8  to  ensure  that  every  student  makes  Adequate  Yearly  Progress 
(AYP).  The  three  schools  in  this  study  are  included  in  this  mandate.  Although  each  school  approached  WIDE  with 
different  goals  and  outcomes  in  mind,  they  all  had  common  interests  in  using  WIDE  courses  to  address  challenges 
identified  in  their  external  assessments  and  to  meet  the  educational  accountability  standards  of  both  NY  State  and 
NYC.  Thus,  as  described  in  this  report,  these  three  principals  purposely  selected  WIDE  courses  and  recruited 
selected  teachers  and  leaders,  with  the  intention  of  meeting  the  specific  accountability  challenges  faced  by  their 
schools. 

This  report  begins  with  an  overview  of  the  Systemic  Educational  Improvement  Model  that  guides  the  design  and 
analysis  of  WIDE  World’s  professional  development  programs,  followed  by  a brief  description  of  the  methods  used 
to  conduct  this  study.  To  situate  this  study  in  its  policy  context,  we  provide  a short  summary  of  the  educational 
reform  priorities  and  some  key  accountability  systems  in  New  York  State  and  NYC.  In  the  next  section,  we 
introduce  each  school  in  this  study,  including  their  goals  and  intended  process  for  working  with  WIDE. 

The  Outcomes  sections  synthesize  salient  themes  and  particular  accomplishments  of  the  three  schools.  They 
summarize  both  the  direct  impact  on  individual  participants  (i.e.,  teachers  and  leaders)  and  the  secondary  impacts  on 
students  and  schools,  with  a particular  focus  on  the  schools’  efforts  to  meet  their  accountability  challenges. 


3 1 have  used  pseudonyms  for  the  names  of  the  three  schools  presented  in  this  study.  I have  not  made  any  other 
changes  to  information  presented  in  the  report  from  each  school. 

4 http://www.ed.gov/policy/elsec/guid/states/index.html  Four  areas  of  NCLB  are  1)  accountability  for  results,  2)  an  emphasis 
on  using  results  from  scientific  research,  3)  expanded  parental  options,  and  4)  increased  local  control  and  flexibility. 
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II.  Research  Context  and  Methods 


Conceptual  Framework:  WIDE  World  Systemic  Educational  Improvement  Model 

Since  its  inception  in  1999,  WIDE  World  has  gradually  evolved  its  goals  from  improving  classroom  practices  of 
individual  teachers  to  foster  broader  improvement  of  educational  systems  over  time.  The  conceptual  framework 
shown  in  Figure  1 depicts  a simplified  and  idealized  overview  of  this  process  and  intended  outcomes. 


We  define  an  educational  system  as  a group  of  educators  working  in  an  organization  (e.g.,  school,  district,  or 
country),  whose  practices  are  influenced  by  a set  of  initial  systemic  conditions  broadly  named  as  Organizational 
Structures,  Educational  Culture,  Educational  Standards  and  Tools  and  Visionary  Leadership.  At  the  beginning  of 
WIDE  World’s  relationship  with  an  institutional  client,  we  collaborate  with  them  to  conduct  informal  assessment  of 
the  initial  conditions  and  co-construct  a plan  for  blended  (online  and  onsite)  professional  development.  We  also 
plan  a means  of  conducting  action  research  with  the  client  to  support  ongoing  assessment  of  the  process  and  impacts 
of  the  professional  development  activity  that  will  inform  further  planning. 

The  first  purple  arrow  next  to  the  orange  circles  identifies  key  characteristics  of  the  professional  development 
programs  supported  by  WIDE  that  help  teachers  and  leaders  to  improve  their  practices.  The  second  purple  arrow,  on 
the  right,  indicates  categories  of  improved  professional  performance  that  we  think  are  key  outcomes  for  teachers  and 
educational  leaders.  These  components  are:  increased  engagement  in  school  work,  flexible  understanding  of  high 
quality  teaching  and  leadership,  and  more  reflective  collaboration  with  colleagues.  These  outcomes  are  displayed  in 
an  arrow  because  we  believe  that  better  performance  by  educators  leads  to  improved  student  performance, 
conceptualized  in  the  same  three  categories:  increased  engagement  with  their  studies,  better  flexible  understanding 
of  their  school  work,  and  more  reflective  collaboration  with  peers.  These  student  outcomes  are  consistent  with  the 
active,  meaningful,  continuous  learning  and  collaboration  that  is  often  described  as  the  necessary  for  student  success 
in  the  21st  century  world. 

The  model  also  indicates  that  improvements  in  performance  by  teams  of  teachers  and  leaders  engaged  in  coherent, 
multi-level  professional  development  ultimately  have  a positive  impact  on  the  systemic  conditions  of  Organizational 
Structures,  Educational  Culture,  Educational  Standards  and  Tools,  and  Visionary  Leadership.  Although  the 
diagram  suggests  change  over  time  in  a left-to-right  progression,  we  believe  that  the  systemic  improvement  process 
is  cyclical.  Improved  systemic  conditions  create  a new  context  for  another  round  of  professional  development  until 
the  client  organization  develops  sufficient  expertise  to  foster  ongoing  learning  and  action  research  that  support  its 
continual  improvement. 
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This  study  of  WIDE’s  professional  development  programs  in  New  York  City  schools  has  allowed  us  to  derive  and 
apply  this  model  to  analysis  of  our  extensive  two-year  research  data.  The  Results  section  of  this  report  organizes 
our  findings  according  to  this  model  to  highlight  the  system-wide  change  process  in  the  three  schools  and  derive 
valuable  crosscutting  themes  from  these  cases. 

Accountability  Systems  in  New  York  State  and  New  York  City 

In  this  section,  we  briefly  explore  ways  in  which  New  York  State  and  NYC  carry  out  their  accountability 
requirements  in  order  to  illuminate  the  policy  context  within  which  these  schools  participated  in  WIDE  World's 
programs. 

In  July  2002,  NYC  Mayor  Bloomberg  appointed  Mr.  Joel  Klein,  a graduate  of  NYC  public  schools,  as  the  first 
Chancellor  of  the  newly-reorganized  Department  of  Education  (DoE).  The  NYC  DoE  serves  over  one  million 
students  in  about  1,500  schools  located  in  Manhattan,  Queens,  Brooklyn,  Bronx,  and  Staten  Island.  According  to  the 
NYC  Department  of  Education  (DoE),  the  Bronx  is  the  third  largest  borough  which  serves  more  than  200,000 
students  as  of  October  2008. 5 

Coming  from  a background  in  government  and  business.  Chancellor  Klein  has  developed  a comprehensive  and 
innovative  education  reform  program.  Children  First,  to  improve  student  academic  performance  as  well  as  to 
“transform  the  nation's  largest  public  school  system  into  a system  of  great  schools.”6  The  Children  First  Initiative 
(CFI),  for  example,  includes  comprehensive  academic  supports  for  most  struggling  students  with  more  high-quality 
educational  choices  (e.g.,  before  or  after  school  tutoring),  and  most  importantly  increased  principals’  authority  over 
how  they  run  their  schools  from  classroom  instruction  to  management,  while  holding  them  accountable  for 
successful  or  failing  results  with  clear  consequences. 

With  accountability  as  its  centerpiece,  the  goal  of  Children  First  Initiative  is  to  “raise  academic  standards,  close 
achievement  gaps,  encourage  more  school  accountability,  and  offer  more  choices  to  families  and  students.”7 

Under  requirements  of  the  federal  No  Child  Left  Behind  (NCLB)  act,  both  the  NY  State  Education  Department 
(SED)  and  NYC  DoE  carry  out  a series  of  formal  and  informal  assessments  each  year  to  identify  every  school’s 
performance  in  relation  to  its  annual  yearly  progress  (AYP)  benchmarks.  Currently  in  NY  State,  AYP  is  determined 
based  on  each  school’s  progress  toward  meeting  the  NY  state’s  proficiency  level  (Score  3)  for  all  students  in  English 
language  arts  (ELL),  mathematics,  science  and/or  high  school  graduation  rate.  If  a school  does  not  meet 
requirements  in  these  assessment  criteria,  it  is  placed  on  the  list  of  “Schools  in  Need  of  Improvement”  for  two  years. 
If  it  makes  no  further  improvement,  the  school  becomes  a “Corrective  Action  School”  and  faces  school  restructuring 
in  the  coming  year. 

In  order  to  accomplish  this  multifaceted  task,  NYC  DoE  uses  a range  of  assessment  mechanisms  such  as  attendance 
data,  school-based  assessment  results  (e.g.,  teacher-developed  evaluations).  Learning  Environment  Survey,  and 
Progress  Report.  Its  annual  Quality  Review  integrates  all  these  activities  by  assigning  a score  on  five  Quality 
Statements. 

Along  with  these  new  standards  and  tools,  the  Children  First  Initiative  also  encourages  teacher-leader  collaborations 
to  implement  school-level  empowerment  and  data-driven  accountability.  The  Inquiry  Teams  (IT),  for  example, 
consist  of  three  to  six  members  in  a school  including  Principal,  teacher  leaders,  and  a Data  Specialist.  Based  on  the 
assessment  data  available  at  their  school.  Inquiry  Teams  identify  a target  group  of  the  lowest  performing  students  in 
a particular  content  area  (e.g.,  7th  grade  students  whose  math  scores  fall  between  1.0  and  2.5  on  NY  State  exams), 
develop  and  implement  differentiated  instructional  practices  to  promote  student  progress,  and  continuously  monitor 
and  share  their  analysis  on  these  students’  academic  improvement  with  the  use  of  accountability  data. 

Because  this  study  examined  the  three  school’s  use  of  WIDE  courses  to  meet  NYC’s  accountability  requirements, 
let  us  briefly  look  at  two  of  NYC’s  particular  accountability  instruments:  Progress  Report  and  Quality  Review. 


5 http://schools.nyc.gov/AboutUs/Data/Stats/default.htm 

6 http://schools.nyc.gov/AboutUs/leadership/chancellorklein.htm 

7 http://schools.nyc.gov/RulesPolicies/NCLB/Overview/default.htm 
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The  Progress  Report  is  performed  to  measure  student  year-to-year  progress,  to  compare  the  school  to  similar  schools 
in  NYC,  and  if  applicable  to  reward  any  success  in  improving  all  children’s  academic  performances,  especially 
children  with  the  greatest  needs  (e.g.,  special  education  students).  It  assesses  each  school’s  progress  on  the  following 
four  areas  of  improvement. 

School  Environment:  uses  results  from  the  Learning  Environment  Survey  to  measure  necessary  conditions  for 
learning:  attendance,  academic  expectations,  communication,  engagement,  and  safety  and  respect. 

Student  Performance:  measures  skill  levels  in  ELL  and  Math. 

Student  Progress:  measures  average  student  improvement  from  last  year  to  this  year  in  ELL  and  Math. 
Additional  Credit:  given  to  schools  for  exemplary  gains  among  high-need  students  to  close  the  achievement 

gap- 

Schools  are  assigned  letter  grades,  from  A to  F,  based  on  their  overall  Progress  Report  scores.  Schools  that  get  As 
and  Bs  are  eligible  for  rewards.  Schools  that  get  Ds  and  Fs,  or  3Cs  in  a row,  face  consequences,  including  change  in 
school  leadership  or  school  closure. 

According  to  NYC  DoE,  the  Progress  Report  results  are  then  integrated  into  its  annual  Quality  Review  which 
evaluates  the  following  areas:  Gather  Data,  Plan  and  Set  Goals,  Align  Instruction,  Build  and  Align  Capacity,  and 
Monitor  and  Revise. 8 

Every  year,  each  school  receives  an  overall  Quality  Review  Score  as  well  as  a separate  score  for  each  of  the  five 
categories  shown  above  in  the  range  of  Well  Developed,  Proficient,  Underdeveloped  with  Proficient  Features,  or 
Underdeveloped.  Together  with  the  Progress  Report,  the  Quality  Review  serves  as  a major  set  of  accountability 
benchmarks  for  principals  to  continuously  assess  all  students’  academic  progress  as  well  as  to  allocate  the  school's 
time,  personnel,  and  resources  for  improvement  results.  What  is  noticeable  here  is  that  the  Quality  Review  criteria 
align  precisely  with  many  aspects  of  WIDE  online  courses  such  as  teaching  for  understanding  and  differentiating 
instruction  courses  (Quality  Statements  2 & 3),  effective  use  and  analysis  of  the  student  data  in  the  Data  Wise  course 
(Quality  Statements  1 & 5),  as  well  as  leadership  focused  on  collaboration  to  improve  instruction  and  student 
performance  in  the  Leading  for  Understanding  course  (Quality  Statement  4). 

While  analyzing  early  impacts  of  WIDE  World  on  the  schools  in  this  paper,  therefore,  we  will  refer  to  the  most 
recent  years’  Quality  Review  scores  of  each  school  as  well  as  how  the  teachers  and  leaders  in  these  schools  have 
made  sense  of  their  WIDE  participation  in  regards  to  their  schools'  accountability  requirements. 

In  the  following  sections,  we  highlight  connections  between  NYC  accountability  requirements  and  WIDE  offerings 
as  we  describe  how  the  three  schools  applied  WIDE  course  ideas  and  experiences  to  their  everyday  practices  to  meet 
external  accountability  requirements. 

Research  Methods 

This  research  project  employed  multiple  data  sources,  both  quantitative  and  qualitative,  and  developed  several 
research  instruments  including  Interview  Guides  for  teachers  and  leaders,  the  HCNY  Codebook  to  systematically 
analyze  the  data,  and  the  HCNY  Coding  Summary  to  examine  emerging  and  cross-school  themes.  The  following 
section  briefly  explains  our  data  sources  and  each  of  the  instruments. 

Interview  Guides  and  Classroom  Observations  ( used  in  research  visits  in  schools) 

In  addition  to  three  site  visits  in  Year  1,  Dr.  Joo  also  visited  the  three  schools  in  November  and  February  of  Year  2 
(2008-09).  In  each  research  visit.  Dr.  Joo  conducted  individual  leader  interviews,  focused  teacher  group  interviews, 
informal  classroom  observations,  and  document  collection. 

In  Year  1,  our  main  site  visit  goal  was  to  carry  out  a series  of  individual  and  focus  group  interviews  to  study  the 
WIDE  participants'  understanding  of  the  goals,  processes,  and  early  impacts  of  WIDE  courses  on  their  teaching  or 
leadership  practices.  In  Year  2,  with  increased  awareness  of  school-wide  changes  in  these  schools,  our  interviews 
complemented  the  original  questions  by  adding  items  on  the  school-based  support  structures  that  influenced 
systemic  educational  improvement,  as  well  as  items  to  identify  first-year  to  second-year  experience  differences.  For 
school  leaders,  we  asked  more  specifically  about  their  visions  for  the  school,  leadership  styles,  and  any  deliberate 


8 http://schools.nyc.gov/Accountability/SchoolReports/QualityReviews/Scores/default.htm 


219 


efforts  they  made  to  create  school-wide  educational  changes  using  lessons  learned  from  their  participation  with 
WIDE.  To  complement  these  interviews,  we  conducted  informal  classroom  observations,  typically  shadowing  the 
Principals  in  each  school. 

Online  Course  Surveys  ( used  in  online  courses) 

Each  WIDE  course  collects  online  data  from  participants  through  pre-,  mid-  and  end-of-course  surveys.  In  each 
survey,  course  participants  answer  both  quantitative  (multiple-choice)  and  qualitative  (open-ended)  questions. 

WIDE  Teaching  Portfolios  (used  as  supplementary  materials) 

As  in  Year  1,  WIDE  distributed  to  eight  teachers  an  updated  version  of  a portfolio  guide  called  "WIDE  World 
Teaching  Portfolio:  What  is  It  Like  to  Learn  in  Your  Classroom  ?"  to  be  used  to  document  and  assess  classroom 
practices  where  WIDE  course  ideas  were  applied.  The  content  of  this  portfolio  includes  the  Portfolio  Instructions, 
Table  of  Contents,  Class  Reflection  Log,  Student  Work  Reflection  Log,  Photograph  Log,  and  the  Student  Consent 
Release  Form. 

In  Year  2,  we  have  received  three  portfolios,  one  from  ECE  and  two  from  TAM. 


All  the  collected  data  were  systematically  analyzed  to  identify  emerging  themes  and  key  findings,  applying  a series 
of  qualitative  analytic  tools  (e.g.,  HCNY  Codebook  & HCNY  Coding  Summary)  developed  throughout  the  two-year 
study  period. 


III.  From  Planning  to  Outcomes  in  Three  Schools 

At  the  beginning  of  this  project,  WIDE  conducted  a selection  process,  requiring  school  leaders  to  submit  a proposal. 
Leaders  were  asked  to  outline  the  way  that  WIDE  professional  development  would  complement  existing  and 
envisioned  efforts  to  improve  instruction  and  student  learning  at  their  school.  Among  the  requirements  of  the 
application  was  that  the  schools  would  assume  responsibility  for  up  to  50%  of  WIDE  World  service  fees,  including 
online  courses  and  on-site  workshops.  This  co-payment  helped  to  insure  that  the  schools  were  invested  in  the 
opportunity.  WIDE  World  invited  proposals  from  participants  in  the  summer  2007  Urban  School  Leaders,  a week- 
long  institute  offered  by  Programs  in  Professional  Education  at  Harvard  Graduate  School  of  Education.  We  also 
contacted  schools  through  the  Children's  Aid  Society's  network  of  Community  Schools  in  New  York.  Of  the  six 
schools  that  were  selected  to  participate  in  Year  1,  three  schools  chose  to  continue  in  Year  2:  two  elementary  (CNE 
& ECE)  and  one  middle  school  (TAM). 

The  table  below  summarizes  the  overall  enrollment  data  per  school  during  Year  1 (2007-8)  and  Year  2 (2008-9): 


School 

Total  # of 
WIDE 
terms 

Total  # of 
WIDE 
courses 

Total  # of 
WIDE  participants 

Total  # of  participants 
with  2 or  more  courses 

Average  rate  of 
course  completion9 

CNE 

4 

7 

16 

(5  admin.  & 1 1 teachers) 

11  (69%) 

100% 

ECE 

4 

4 

21 

(5  admin.  & 16  teachers) 

11  (52%) 

81% 

TAM 

3 

5 

27 

(7  admin.  & 20  teachers) 

16  (59%) 

72% 

[Table:  HCNY  Overall  Enrollment  Data  per  School,  Fall  2007-Spring  2009] 


Each  school  and  its  principal  had  responsibility  for  planning  particular  implementation  steps  for  WIDE  participation, 
and  recruiting  administrators  and  teachers  to  particular  WIDE  courses. 


9 To  complete  a course,  participants  must  carry  out  32  out  of  45  possible  course  hours  (used  to  determine  Professional 
Development  Points)  over  the  six  sessions  of  the  course. 
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The  following  section  introduces  each  school  in  the  following  structure: 

School  mission  and  current  educational  priorities 

School  demographics  data  based  on  the  NYC  Quality  Review  2007-08 

Specific  goals  for  taking  particular  WIDE  courses 

The  selection/recruitment  process  of  participants 

The  summary  of  the  enrollment  data  with  WIDE  per  term 


Connected  Neighbor  Elementary  School  (CNE  - Principal:  Ms.  Jones) 

CNE  serves  students  in  pre-k  to  fifth  grades.  CNE’s  mission  is  “to  promote  learning  for  all  students  in  an  active, 
respectful  and  skill-building  environment.  We  believe  that  every  child  is  important  and  valued.  Our  students  will 
become  self-confident,  self-motivated  and  self-reliant  learners.”  (CNE  Fact  Sheet,  2007)’° 

More  specifically,  CNE  aims  to  teach  problem  solving  and  critical  thinking  skills,  promote  self-esteem  and  nurture 
love  of  learning  among  the  students  as  lifelong  learners.  CNE  also  emphasizes  learning  for  all  students  in 
collaboration  with  parents,  community,  and  community-based  organizations.  In  2002,  CNE  began  its  partnership 
with  the  Children’s  Aid  Society,  a non-profit  organization  that  operates  twenty-one  community  schools  in  New 
York  City,  to  develop  after-school  programs  and  to  promote  collaboration  among  NYC  schools,  the  families  of 
students,  and  the  wider  gamut  of  community  services,  including  the  Bronx  Family  Center.*  1 1 

A Bronx  native,  Principal  Jones  has  worked  in  the  NYC  school  system  for  over  25  years  and  served  as  a teacher, 
speech  therapist,  special  education  supervisor,  assistant  principal,  and  since  1998,  Principal  of  CNE. 

The  following  table  summarizes  the  overall  enrollment  data  at  CNE  in  the  year  of  2007-08. 


CNE  (2007-08) 

Total  Enrollment 

371  (PreK-5  grades) 

- by  gender 

52%  Male;  48%  Female 

- by  race/ethnicity 

69%  Hispanic,  29%  Black,  1%  White,  1%  Asian 

- Special  education  students 

16% 

- English-language  learners 

10% 

- Eligibility  for  free  lunch 

83% 

Attendance 

93% 

[Table:  CNE  Enrollment  Data] 


In  terms  of  educational  priorities,  CNE  as  an  urban  community  school,  was  facing  two  immediate  goals:  to  provide 
high  quality  education  to  meet  different  student  needs  and  to  use  the  student  assessment  data  to  improve  its  NYC 
Quality  Review  results. 

According  to  the  2006-07  Quality  Review,  CNE  was  identified  as  “in  need  of  improvement  - Year  1”.  This  meant 
that,  if  the  school  did  not  improve  their  accountability  status  in  the  next  year,  it  would  face  serious  consequences, 
including  a change  in  school  leadership  or  possible  school  closure.12  The  Quality  Review  said  that  “CNE  teachers 
are  not  differentiating  learning  enough  to  affect  the  student  outcome,  and  students  who  are  performing  well  are  not 
getting  the  attention  that  they  need.”  The  Review  continued  that  “many  students  are  not  actively  involved  in  their 
own  learning.  They  are  not  sure  of  what  they  are  learning  and  why  and  how  they  can  improve  their  work.”  In  our 
first  interview  with  Principal  Jones  during  Year  1,  she  confessed  that  this  review  was  a complete  shock  to  her.  She 
believed  that  her  school  was  doing  well  and  meeting  all  areas  of  accountability,  including  student  outcomes. 


10  Accessed  from  the  CNE  School  website. 

11  http://www.childrensaidsociety.org/communityschools 

12  Information  from  the  NYC  DoE's  Progress  Report. 
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Upon  reflection.  Principal  Jones  concluded  that  the  low  performance  was  mainly  due  to  the  school's  inability  to 
collect,  examine,  and  present  student  assessment  data  effectively  to  the  Quality  Reviewer.  Therefore,  for  the  2007- 
OS  school  year,  she  set  a clear  priority  to  increase  the  leadership  capacity  within  the  school  as  well  as  to  develop  a 
comprehensive  and  systemic  way  to  collect,  catalog,  and  analyze  student  assessment  data  (the  major  component  of 
the  Quality  Review).  At  the  same  time,  she  wanted  her  teachers  to  learn  more  about  effective  curriculum 
development  and  the  major  principles  and  practices  of  differentiating  instruction. 

In  our  first  interview  during  Year  1,  Principal  Jones  explained  how  carefully  she  selected  WIDE  courses  to  match 
her  school's  goals  and  then  identified  participants  for  those  courses.  For  example,  she  explained  that  she  selected 
aspiring  school  leaders,  experienced  teachers,  junior  teachers,  and  staff  of  the  Children's  Aid  Society  to  take  courses 
on  Teaching  for  Understanding  (TfUl)  and  Leadership  (LfUl)  during  the  first  term  with  WIDE  (Fall  2007)  because 
she  wanted  to  build  a collaborative  team  among  teachers  and  potential  leaders  to  deal  with  “school-wide  issues.” 
(CNE  FG4,  69-73).  For  the  next  term  (Spring  2008),  her  leadership  team,  including  her,  enrolled  in  the  Data  Wise 
(DW)  course  while  a different  group  of  teachers  took  the  Differentiating  Instruction  (DI)  course. 

For  the  Datawise  course.  Principal  Jones  assigned  an  administrator  as  Data  Specialist,  whose  nickname  became 
“Data  Guru”  or  “Data  Dude”,  among  the  CNE  faculty.  This  Data  Specialist  worked  closely  with  Principal  Jones  and 
the  teachers  to  create  various  data  sets  from  student  assessments,  including  NY  State  and  NYC  exam  results.  Using 
Excel,  he  was  able  to  produce  comprehensive,  reader-friendly  tables  and  charts.  According  to  Principal  Jones, 
“Anything  that  the  teachers  need,  he  has  it!”  In  addition,  the  specialist  provided  a series  of  professional 
development  activities  for  teachers  that  made  use  of  the  school  data. 

Along  with  recruiting  key  school  personnel  to  WIDE  courses.  Principal  Jones  designated  every  Thursday  as  an  after- 
school period  for  WIDE  participants  to  work  together  in  the  school  library. 

Thanks  to  Principal  Jones’s  careful  selection,  a majority  of  the  first  year  cohort  have  stayed  together  to  take  a series 
of  WIDE  courses  in  Year  2CNE  and  Jones  other,  new  teachers  and  administrators  joined  their  ranks  in  that  second 
year. 

Principal  Jones  set  up  an  open  enrollment  policy  where  teachers  could  feel  comfortable  making  their  own  decisions 
to  continue  with  the  program  or  not.  “1  don’t  want  anybody  to  feel  that  if  I asked  them  that  they'd  have  to  do  it  or  I'd 
be  holding  something  against  them.”  [CNE  Jones2]  Two  WIDE  participants  echoed  this  open  enrollment  process  by 
explaining  that  “Ms  Jones  invites  us,  but  we  choose  if  we  want  to  stay  or  not”  [CNE  FG1_MI]  and  “it  [recruitment] 
is  definitely  our  choice  and  we  saw  the  value  in  it  when  we  did  the  first  one,  and  so  we  have  continuously  done  this. 
However,  she  has  made  it  open  to  other  people”.  [CNE  FG1_MI] 

Since  the  fall  of  2007,  a total  of  16  CNE  participants  (5  administrators  and  1 1 teachers)  have  taken  seven  WIDE 
courses  over  four  terms,  as  shown  below.  Eleven  out  of  sixteen  participants  took  WIDE  courses  for  more  than  two 
terms,  for  a 69%  ( 1 1/16)  ongoing  participation  rate,  and  five  participants  including  Principal  Jones  took  four 
courses,  one  course  per  term.  The  course  completion  rate  in  this  school  is  very  impressive. 
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WIDE  term 

WIDE  course13 

Total  # of  enrollment 

Course  Completion  Rate 

Fall  07 

TfUl 

8 

100% 

LfUl 

5 

100% 

Spring  08 

DI 

4 

100% 

DW 

5 

100% 

Fall08 

MI 

3 

100% 

TSNT1 

6 

100% 

Spring09  (current  term) 

DI 

3 

N/A 

TfU2 

6 

N/A 

[Table:  CNE  overall  enrollment  in  WIDE  courses,  2007-09] 


Every  Child  Elementary  School  (ECE  - Principal:  Mr.  Miller) 

ECE’s  stated  mission  is  to  “create  a community  of  lifelong  learners  in  pursuit  of  academic  excellence,  where  all 
students  learn  the  necessary  skills  to  become  caring,  creative  and  productive  citizens.  We  will  provide  a standards- 
driven  curriculum  that  targets  the  different  learning  styles  of  all  our  students,  including  special  needs,  high  achieving 
and  ELL  students.  We  will  involve  community,  parents,  staff  and  students  in  maintaining  high  levels  of  participation 
and  achievement  in  an  ever-changing  global  society.”14  According  to  the  school’s  2008-09  Comprehensive 
Educational  Plan,  ECE  concentrates  its  efforts  around  clear  and  high  expectations  for  teachers  and  students, 
meaningful  assessment  aligned  with  the  NY  State  standards,  and  effective  data-driven  professional  development 
based  upon  the  differentiated  needs  of  students  and  teachers. 

With  nearly  thirty  years  of  experience  in  the  NYC  public  education.  Principal  Miller  came  to  ECE  in  2003.  At  that 
time,  the  school  was  about  to  be  closed  due  to  low  performance,  and  so  Principal  Miller  started  to  create  new 
services,  reach  out  to  parents,  and  secure  grants  to  fill  budget  gaps.  According  to  the  NYC  2006-07  Quality  Review, 
this  school  “has  made  dramatic  improvements  in  achievement  and  attendance  while  reducing  behavior  incidents” 
since  the  current  administration  took  charge. 

Located  in  three  separate  buildings,  ECE  serves  over  580  students  as  shown  in  the  following  enrollment  table. 


ECE  (2007-08) 

Total  Enrollment 

583  (PreK-5  grades) 

- by  gender 

52%  Male;  48%  Female 

- by  race/ethnicity 

73%  Hispanic,  22%  Black,  3%  Asian,  2%  White 

- Special  education  students 

13% 

- English-language  learners 

12% 

- Eligibility  for  free  lunch 

71% 

Attendance 

93% 

[Table:  ECE  Enrollment  Data] 


Although  ECE’s  2006-07  Quality  Review  gave  an  overall  rating  of  “well-developed  school,”  it  also  identified  that 
the  school  needed  to  “increase  professional  development  regarding  the  use  of  data  for  differentiating  instruction  for 
individual  students”  and  “devise  collaborative  practices  and  structures  to  support  the  vertical  alignment  of 
curriculum  and  instruction  in  all  content  areas.” 


13 

The  full  name  of  each  course  is  as  follows. 

- Dl  (Differentiating  Instruction:  Strategies  for  Effective  Classroom  Practice) 

- Ml  (Using  Multiple  Intelligences  as  a Tool  to  Help  Students  Learn) 

- TfUl  (Teaching  for  Understanding  1:  Focus  on  Student  Understanding) 
Technologies  1) 

- TfU2  (Teaching  for  Understanding  2:  Understanding  in  Practice) 

14  Accessed  from  the  ECE  School  website. 


DW  (Getting  Started  With  Data  Wise) 
LfUl  (Leading  for  Understanding  1) 
TSNT1  (Teaching  to  Standards  with  New 
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In  the  first  term  with  WIDE  (Fall  2007),  Principal  Miller  selected  teachers  whom  he  thought  would  benefit  from 
WIDE’s  Data  Wise  course.  As  he  reflected  upon  his  mandatory  enrollment  decision  later,  he  realized  that  “they  [the 
first  cohort  whom  he  assigned  by  himself]  were  not  interested  in  taking  the  program.  Some  of  them  had  problems 
with  the  use  of  computer;  some  of  them  said  it  was  other  people’s  job  to  keep  the  data.  As  shown  below,  the  course 
completion  rate  in  the  first  term  was  57%. 

In  his  second  year.  Principal  Miller  simply  announced  the  WIDE  courses  to  all  teachers  and  administrators,  and 
allowed  them  to  decide.  If  teachers  were  interested,  he  encouraged  them  to  sign  up  for  a course  as  a team,  in  order  to 
promote  collaboration  at  the  school  level.  “Now  [the  school  people  are]  free  [to  choose]  and  people  who  want  to 
study,  just  sign  up.  And  it  is  much  better,”  said  Miller. 

Since  ECE  had  been  already  successful  in  data-driven  decision  making  with  respect  to  instruction  and  overall 
management  in  Year  2,  Principal  Miller’s  particular  priority  for  WIDE  was  to  “meet  the  needs  of  all  our  learners”  by 
involving  all  13  WIDE  participants  in  the  Differentiating  Instruction  ( DI)  course  in  order  to  strengthen  “a  vertical 
alignment  of  such  instructions  across  K-5  grade  levels.” 

Like  Principal  Jones  at  CNE,  Principal  Miller  arranged  teaching  schedules  to  reserve  a common  meeting  time.  The 
ECE  WIDE  participants  met  Monday  afternoons  to  work  together  on  the  DI  course,  and  that  continued  throughout 
the  term. 

The  following  table  ECE  shows  the  enrollment  and  course  completion  data  for  ECE  over  the  last  two  years. 


WIDE  term 

WIDE  course15 

Total  # of  enrollment 

Course  Completion  Rate 

Fall  07 

DW 

7 

57% 

Spring  08 

LfUl 

8 

100% 

Fall08 

DI 

15 

87% 

Spring09 

TfUl 

6 

100% 

[Table:  ECE  overall  enrollment  in  WIDE  courses,  2007-09] 


Technology  and  the  Arts  Middle  School  (TAM  - Principal:  Ms.  Ellis) 

The  mission  for  TAM,  established  in  2001,  states  that  TAM  aims  “to  provide  a standards-based  curriculum,  to  create 
a school  community  dedicated  to  promoting  continuous  learning  and  achievement  through  technology,  and  to 
nurture  and  prepare  our  community  to  adapt  to  meet  the  demands  of  an  ever  changing  world.”  By  nurturing  and 
enhancing  students'  learning,  TAM  teachers  and  leaders  want  their  students  to  understand  the  connection  between 
what  they  leant  in  TAM  and  the  skills  needed  in  the  real  world  through  the  use  of  technology. 

As  a middle  school  located  in  a challenging  neighborhood  in  Bronx,  TAM  serves  364  students  from  grade  6 to  8. 
Note  that  the  eligibility  level  for  Title  1 funding  among  TAM  students  is  almost  90%,  much  higher  than  the  average 
in  the  area. 


15 

The  full  name  of  each  course  is  as  follows. 

- DI  (Differentiating  Instruction:  Strategies  for  Effective  Classroom  Practice) 

- TfUl  (Teaching  for  Understanding  1:  Focus  on  Student  Understanding) 


DW  (Getting  Started  With  Data  Wise) 
LfUl  (Leading  for  Understanding  1) 
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TAM  (2007-08) 

Total  Enrollment 

364  (6-8  grades) 

- by  gender 

51%  Male;  49%  Female 

- by  race/ethnicity 

64%  Hispanic,  34%  Black,  1%  White,  1%  Asian 

- Special  education  students 

21% 

- English-language  learners 

15% 

- Eligibility  for  free  lunch 

87% 

Attendance 

85% 

[Table:  TAM  Enrollment  Data,  2007-08] 


In  2006-7,  the  year  before  WIDE  collaboration  started,  TAM’s  Progress  Report  stated  it  was  a school  with  “D: 
Planning  for  Restructuring”  grade  and  its  previous  year’s  NY  State  performance  review  designated  the  school  as  “In 
Corrective  Action.”  Although  there  was  no  narrative  data  available  in  these  reports  to  describe  the  issues  at  the  time, 
our  first  year  research  experience  at  TAM  offered  a glimpse  of  various  difficulties  with  school-related  issues  and 
out-of-school  challenges  (some  students  from  very  difficult  family  and  community  surroundings).  In  one  of  our 
interviews  during  Year  1,  for  example,  a staff  member  commented  that  they  struggle  with  serious  matters,  such  as 
student  pregnancy,  drug  use  and  prostitution,  in  addition  to  the  more  typical  issues  of  disciplinary  problems  and  high 
student  turnover,  common  to  many  urban  middle  schools. 

Despite  those  issues,  the  2006-07  Quality  Review  actually  labeled  TAM  as  “a  proficient  school  with  many  well 
developed  features”,  including  the  use  of  data  to  drive  all  aspects  of  development  as  well  as  the  existing  school 
structures  in  place  to  improve  student  outcomes.  The  Review  identified  three  main  areas  of  improvement  for  TAM: 
to  make  greater  use  of  high  levels  of  educational  technology  to  support  learning,  to  differentiate  teachers’  instruction 
more  consistently  and  rigorously  based  on  data,  and  to  ensure  that  “all  teachers  complete  the  move  from  teacher- 
centered  instruction  to  facilitative  learning.” 

For  the  first  term  with  WIDE  (fall,  2007),  Principal  Ellis  enrolled  all  of  the  school's  permanent  staff  in  Teaching  to 
Standards  with  New  Technologies  1 (TSNT1).  Due  to  timing  issues,  she  was  not  able  to  inform  teachers  about  the 
course  until  they  reported  to  work  on  their  first  day  of  school.  Forty  percent  of  the  teachers  dropped  out  of  the  online 
course  early,  and,  of  those  who  remained  in  the  course,  18  of  27  (67%)  completed  the  course.  From  our  interviews 
with  administrators  and  teachers  at  TAM,  it  was  evident  that  the  low  participation  and  completion  rates  reflected  a 
lack  of  communication  and  the  absence  of  common  understanding  among  school  members. 

In  the  second  term  of  Year  1 (spring,  2008),  participating  TAM  faculty  divided  into  two  groups,  each  with  much 
smaller  enrollment.  The  leadership  group,  including  Principal  Ellis,  took  Data  Wise  and  the  teacher  group  took 
Differentiating  Instruction. 

One  of  the  lessons  learned  from  this  first-year  experience  was  the  importance  of  organizing  professional 
development  so  that  it  systematically  addresses  both  the  internal  needs  of  the  school  and  the  external  accountability 
requirements.  Internally,  TAM  needed  to  improve  its  communication  patterns  between  the  administration  and 
teachers,  between  teachers  and  students,  as  well  as  among  teachers.  Externally,  TAM  needed  to  strengthen  its 
technology  integration  into  the  curricula  in  order  to  provide  engaging  and  effective  instruction  for  all  students  and  to 
monitor  and  improve  the  student  assessment  data  on  an  ongoing  basis. 

In  Year  2,  the  leadership  group  moved  into  the  Leading  for  Understanding  1 course,  while  the  teacher  group 
enrolled  in  the  Teaching  to  Standards  with  New  Technologies  2 (TSNT2)  to  build  their  expertise  in  curriculum 
technology  integration.  Thanks  to  this  coordinated  effort,  the  following  year’s  Progress  Report  assigned  TAM  with  a 
“C:  Well-developed”  grade. 
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WIDE  term 

WIDE  course16 

Total  # of  enrollment 

Course  Completion  Rate 

Fall  07 

TSNT1 

27 

67% 

Spring  08 

DI 

6 

67% 

DW 

6 

100% 

Fall08 

LfUl 

8 

100% 

TSNT2 

3 

33% 

[Table:  TAM  overall  enrollment  in  WIDE  courses,  2007-08] 


Taking  into  account  the  unique  goals  and  expectations  of  each  school,  the  following  Results  section  highlights 
salient  themes  across  all  three  schools  and  celebrates  particular  accomplishments  of  each  school. 


IV.  Outcomes  for  Teachers  and  Leaders 

Our  Year  1 research  indicated  that  schools,  which  met  at  least  two  of  the  following  three  conditions,  appeared  best 
equipped  to  take  full  advantage  of  WIDE  World  professional  development  programs: 

• The  school  had  a coherent  educational  focus  that  aligned  with  WIDE  World  content 

• The  school  had  intact  professional  learning  communities,  bolstered  by  the  collaborative  process  of 
participating  in  WIDE  courses 

• School  leadership,  whether  the  Principal  or  the  leadership  group,  had  a clear  vision  and  support  structures 
in  place  to  facilitate  the  integration  of  WIDE  professional  development  as  a process  as  well  as  pedagogical 
content 

The  second  year’s  research  confirmed  the  importance  of  these  conditions  and  all  three  schools  showed  improvement 
during  Year  2,  due  to  their  continuing  participation  with  WIDE. 

As  introduced  in  the  Research  Context  section,  the  WIDE  Systemic  Educational  Improvement  Model  ( SEIM)  is 
used  as  an  analytic  framework  to  conceptualize  detailed  examples  of  both  individual-  and  school-level  changes. 
Because  WIDE  courses  are  offered  to  teachers  and  leaders,  we  will  begin  with  direct  impacts  on  the  professional 
practice  of  teachers  and  leaders.  In  the  next  section,  we  will  trace  the  indirect  impacts  of  improved  practices  on 
student  performance  and  on  the  overall  school  system. 

In  this  section.  Outcome  for  Teachers  and  Leaders,  findings  are  presented  in  the  following  three  categories  of  the 
SEIM  model: 

Understanding  for  teachers  and  leaders'.  The  capability  to  apply  one's  knowledge  creatively  and 
appropriately  in  a range  of  everyday  situations  at  work. 

Engagement  of  teachers  and  leaders : Enthusiasm,  interest  in,  commitment  to,  and  caring  about  the  work 
one  does  at  school.  Although  it  may  contribute  to  improved  performance,  this  category  refers  more  to 
attitudes  and  beliefs  regarding  one's  identity  as  a professional,  whereas  the  understanding  category  refers 
to  patterns  of  practice. 

Reflective  collaboration  for  teachers  and  leaders'.  The  capability  to  participate  in  inquiry  that  includes 
analyzing  and  attempting  to  improve  one's  work  in  relation  to  explicit  criteria  or  definitions  of  high- 
quality  practice.  Collaboration  is  a capability  and  desire  to  converse  and  work  together  effectively  with 
one's  colleagues  within  and  across  system  levels  to  achieve  the  common  goals  of  systemic  improvement. 

Within  each  category,  findings  will  be  presented  for  each  school  (CNE,  ECE,  and  TAM). 


16  The  full  name  of  each  course  is  as  follows. 

- Dl  (Differentiating  Instruction:  Strategies  for  Effective  Classroom  Practice)  - DW  (Getting  Started  With  Data  Wise) 

- TSNT1  (Teaching  to  Standards  with  New  Technologies  1)  - LfUl  (Leading  for  Understanding  1) 

- TSNT2  (Teaching  to  Standards  with  New  Technologies  2) 
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Understanding 

1.  CNE 

One  teacher  at  CNE  explained  that  a lot  of  teachers  in  her  school  thought  that  their  job  was  “just  collecting  the  data” 
without  knowing  “what  that  really  meant  or  what  to  do  with  it.”  While  taking  the  Data  Wise  course,  she  said  that 
“we  [WIDE  participants]  started  to  see,  really  see,  kids  who  were  not  moving  or  suddenly  start  to  move  because  of 
the  decisions  I made  based  on  my  [DW]  information.  That’s  when  you  start  to  see  the  teachers  saying,  ‘Ah!  It 
works!”’  [CNE  FG1_MI]  Another  teacher  at  CNE  pointed  out  that  “DW  was  sort  of  a launching  pad  for  us  to  get 

our  data  going....  The  quality  reviewers  were  there,  but  we  didn't  know  how  [to  carry  out  the  quality  review] I 

think  the  DataWise  course  helped  us  to  see  how.  It  gave  us  a how  to  do  it,  and  a focus.  ...  So  the  DataWise  course 
helped  put  us  on  a good  track:  what  we  need  to  focus  on,  how  we  can  probably  get  together,  and  work  this  through.” 

According  to  a teacher  who  took  the  Multiple  Intelligences  (MI)  course,  “the  MI  course  helps  me  to  be  able  to 
appreciate  where  our  teachers  are  coming  from,  because  sometimes  you  want  to  be  able  to  reach  them.  We  reach  our 
children  at  their  levels,  yes,  but  it’s  sometimes  more  difficult  to  get  buy-in  from  the  teachers.  The  multiple 
intelligences  course  helps  you  to  sort  of  understand  where  people  are  coming  from,  why  they  are  the  way  they  are, 
and  why  they  may  think  the  way  they  think.”  [CNE  FG1_MI] 

After  taking  her  first  WIDE  course,  one  teacher  reflected  upon  her  experience  in  view  of  the  state  exams  her 
students  were  taking.  “Prior  to  this  course,  I thought  that  if  my  students  were  able  to  answer  questions  on  a test,  it 
meant  that  they  learned  the  topic  taught.  I now  understand  that  true  understanding  is  not  something  that  is  measured 
by  a specific  test.  Now  when  I teach  a unit,  I look  for  a variety  of  measures  to  assess  the  students  with.  I want  to 
know  that  they  really  comprehend  on  a variety  of  levels.”  [End-of-course  online  survey,  Fall07,  CNE-TfUl,  11547] 

Regarding  to  the  Differentiating  Instruction  ( DI)  course,  CNE  teachers  offered  concrete  examples  of  how  they 
combined  DI  strategies  with  ideas  from  other  WIDE  courses  to  improve  students’  understanding.  A pair  of  teachers 
who  took  TSNT  1 and  DI  together,  explained  that  “we  incorporate  technology  into  the  classroom  and  we’re  able  to 
differentiate,  like  you  could  go  to  different  websites,  have  the  kids  use  different  tools,  calculators,  all  sorts  of  visual 
aids  like  Power  Point,  the  Smart  Boards.  It  shows  that  kids  can  actually  make  more  connections  and  they  can 
actually  get  more  out  of  the  lesson  by  bringing  technology  into  the  classroom.”  [CNE  FG2_TSNT1] 

2.  ECE 

Teachers  at  ECE  provided  abundant  examples  of  how  the  DI  course  helped  them  in  their  classroom  teaching.  One 
teacher  explained  that  “I  think  the  philosophy  is  that  students  should  know  their  individual  strengths  and  abilities 
and  should  take  responsibility  for  their  own  learning. . . starting  in  the  first  grade  where  I teach.  I think  it’s  important 
as  kids  go  to  second  grade  that  they  know  what  their  strengths  are  and  what  their  needs  are  and  who  to  go  to  for  help 
and  where  they  can  be  responsible  for  their  work.”  [ECE_FG]  Another  teacher  described  her  effort  in  carrying  out 
her  writing  initiatives  as  “a  differentiating  activity.”  She  continued,  saying  that  “when  I sit  with  a student  and  pre- 
assess where  they  are  and  write  observation  notes  of  what  they  can  do  in  terms  of  their  letters,  sounds  and  writing  to 
determine  the  teaching  point,  I assess  my  students  in  their  individual  abilities  and  I try  to  understand  where  they  are 
coming  from,  what’s  their  style,  how  their  letters  are  forming,  what’s  their  focus  so  in  the  pre-assessment,  the 
designing  of  the  writing  task.  In  the  post-assessment,  I always  look  at  the  student’s  individual  learning.”  [ECE_FG] 

Another  teacher  appreciated  the  iterative  way  that  WIDE  course  activities  encourage  classroom  applications,  which 
was  very  helpful  for  her  understanding.  She  said  that  “what  we  learned  about  differentiation  [in  the  online  course]  is 
to  connect  it  to  the  way  we  tried  it  in  the  classroom.  We  [WIDE  participants]  had  a larger  question  about  the 
differentiation  and  tried  it  in  the  classroom  and  got  back  to  the  [online]  course.  That  was  very  helpful-  to  have  a 
chance  to  make  mistakes  and  step  back  and  try  things  again.  The  technique  of  differentiation  - you  have  to  try  it  in  a 
small  group  first  and  then  try  it  on  a whole  class  and  that’s  what  we  did  to  understand.”  [ECE_FG] 

In  the  End-of-course  online  survey,  one  ECE  teacher  described  the  ways  she  applied  the  WIDE  course  concepts  to 
her  social  studies  unit  on  New  York  City.  “Before  the  [WIDE]  course,  I would  have  most  likely  just  done  a lecture  - 
having  all  the  students  read  the  same  material  and  then  gather  to  discuss  what  they  found.  However,  what  I have 
decided  to  do  is  to  differentiate  the  instruction  not  just  by  providing  various  activities  to  address  Multiple 
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Intelligences  but  also  to  ask  myself  as  the  facilitator  ‘why  am  I doing  this’  and  ‘what  is  it  that  I want  the  students  to 
learn’ . I have  not  finished  the  unit  yet  but  it  has  worked  very  well  and  most  importantly  the  students  understand 
what  they  are  doing  and  why.”  [End-of-course  online  survey,  Fall08,  ECE-DI,  15057] 

During  the  research  visit  to  ECE  in  November  2008,  Principal  Miller  and  Dr.  Joo  observed  a Kindergarten  class, 
taught  by  one  of  the  DI  course  teachers.  As  the  picture  below  shows,  her  students  were  divided  into  different  groups, 
based  on  their  interests,  during  a science  lesson.  The  25  students  were  busy,  doing  their  own  hands-on  experiments 
in  different  corners  of  the  classroom,  while  the  teacher  circulated  and  worked  with  individual  students  or  groups. 
After  our  observation.  Principal  Miller,  who  conducts  frequent  informal  visits  to  classrooms,  applauded  that  teacher 
by  saying  “I  am  just  proud  to  see  how  [the  teacher]  was  teaching  her  Kindergartners.  That  was  DI  at  its  best.” 
[ECE_Miller] 
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[Picture:  ECE  teacher’s  display  on  student  groups  in  a Kindergarten  class] 


Administrators  in  school  leadership  positions  also  expressed  changes  in  their  understanding  of  what  was  going  on  in 
the  school.  One  of  the  ECE  administrators  wrote  that  “as  an  administrator  of  the  school,  things  that  I would  not 
ordinarily  think  about  in  the  course  of  a school  day  were  brought  to  my  attention  by  the  teachers.  The  [WIDE]  team 
was  helpful  in  rolling  out  the  concept  of  differentiation  to  the  rest  of  the  school,  and  served  as  providers  of 
information  to  others  with  similar  classrooms  and  classroom  needs.”  [End-of-course  online  survey,  Fall08,  ECE-DI, 
10833] 


In  the  individual  interview,  Principal  Miller  explained  that  “as  someone  going  around  the  school  and  seeing  its 
complexity,  it’s  been  very  interesting  to  me  to  observe  this  ]DI]  lesson  and  see  the  course  working,  see  how  kids 
were  divided  according  to  their  strengths-  someone  was  writing,  someone  was  doing  research  activity,  someone 
presenting,  it  was  really  great  to  see  this  work.  DI  is  so  well  embedded  already  in  their  [his  teachers]  philosophy  of 
teaching.  They  don’t  even  realize  that  they  are  doing  [DI],  . . . When  you  are  a really  good  teacher,  sometimes,  it  is 
very  difficult  to  articulate  the  work  that  you  do  in  the  classroom,  you  just  do  it.”  [ECE_Miller] 

3.  TAM 


Referring  to  the  change  in  her  leadership  style  from  “dictatorship  to  understanding-ship,”  Principal  Ellis  shared  a 
story  about  her  changed  understanding  of  her  role  as  a principal.  “When  I came  here  [TAM],  it  [my  leadership]  was 
like  a dictatorship.  It  was  more  like  top-down.  Some  of  the  teachers  resisted.  Some  didn’t  say  anything,  but  they 
wouldn’t  do  anything  either  . . . passive-aggressive,”  Principal  Ellis  confessed.  After  taking  the  Leading  for 
Understanding  1 (LfUl)  course  with  her  leadership  team.  Principal  Ellis  explained  some  beginning  changes  that 
occurred  to  her.  “It  is  helping  me  to  analyze  myself  in  every  decision  that  I make.  I’m  always  reflecting.  I always 
take  into  account  that  decision  is  for  the  best  of  the  children  and  the  school.”  [TAM  Ellis  Nov20]  She  continued  that 
“I  guess  leading.  ...I  do  have  to  communicate  with  people.  And  I would  like  them  to  be  able  to  hear  what  I’m  saying 
and  I’d  like  to  hear  what  they’re  saying  instead  of  being  a dictator.  It’s  all  about  understanding  and  communication. 
And  just  making  sure  we’re  all  on  the  same  page  and  something  is  done,  but  not  done  because  I’m  making  you  do 
it. ..just  because  you  want  to  do  it...  to  internalize.”  [TAM  Ellis2] 
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Ellis  believes  teacher  accountability  is  very  important  and  has  learned  to  seek  it  in  a more  nurturing  and  a caring 
way  by  listening  and  questioning  her  teachers.  When  something  is  amiss,  she  said  that  she  wonders  first  whether 
there  was  a misunderstanding  and  “tries  to  see  things  from  their  points  of  view.'”’It  doesn’t  have  to  be  a 
dictatorship...  I can  lead  through  questioning,”  Principal  Ellis  explained. 

However,  there  were  some  course  participants  who  reported  difficulties  in  WIDE  courses,  especially  due  to  the 
wide-range  of  roles  (teachers  vs.  administrators),  teaching  subjects,  and  grades  among  the  course  participants. 
According  to  one  administrator,  the  unbalanced  course  content,  which  is  geared  toward  classroom  teachers  or 
leaders,  made  it  difficult  for  her  to  understand  and  participate  in  the  curriculum  development  activities  with  her  team 
members.  [End-of-course  online  survey,  Spring08,  CNE-DI,  11575] 


Engagement 
1.  CNE 

One  of  the  key  teaching  strategies  promoted  by  WIDE  courses  is  to  move  teachers  from  being  lecturers  to  become 
facilitators  of  student  learning.  Many  teachers  in  our  study  shared  their  experiences  modeling  this  facilitator  role. 
One  teacher  said  that  “what  I’ve  noticed,  instead  of  being  a teacher  all  the  time,  I could  be  the  facilitator,  where  the 
kids  can  actually  teach  themselves,  and  they  can  assess  themselves.  Like  aligned  with  the  reflection  pieces,  the  kids 
are  asking  themselves  what  do  they  learn,  what  do  they  need  to  improve  on.  And  basically  they’re  being  more 
accountable  to  their  learning.”  [CNE  FG2_TSNT1] 

The  increased  level  of  engagement  with  students  and  their  academic  achievement  also  made  a teacher  feel  “more 
professional  after  I started  using  the  data.”  “...Because  when  I sit  down  with  parents,  instead  of  just  saying,  well, 
your  child  is  struggling  I want  him  to  read  more,  I could  actually  say,  ‘your  child  is  having  difficulty  analyzing  the 
main  idea.  What  he  needs  to  do  is  this.’  And  it  gave  me  real  usable  information,  and  I can  really  tell  their  parents, 
‘This  is  why  your  kid  is  having  a hard  time.’  It  was  concrete  and  it  made  me  feel  like  a professional.  Because  before 
that,  I couldn’t  necessarily  articulate ” [CNE  FG2_TSNT1] 

One  CNE  teacher  indicated  that  WIDE  courses  helped  her  with  the  challenge  of  balancing  student  engagement  with 
ultimate  accountability  for  the  material.  “I  think  one  of  the  things  that  sometimes  we  [teachers]  struggle  with  is  that 
we’re  given  curriculum,  we’re  told  to  make  it  fun,  make  it  interesting.  But  we  have  to  teach  it,  and  we  have  to  cover 
so  many  standards  and  so  many  topics.  If  we  don’t  make  it  meaningful  for  the  kids,  then  it’s  just  us  talking  and  them 
not  really  getting  it.  And  1 think  that  this  [TSNT  course]  gives  us  a way  to  kind  of  bring  them  [students]  in,  and 
reminds  me  to  include  them  in  the  thinking  and  the  planning  and  the  doing  [of  my  lessons].”  [CNE  FG2_TSNT1] 

This  heightened  sense  of  professionalism  was  also  shared  with  Principal  Jones  who  took  two  years  of  WIDE 
courses.  She  confessed  that  “I  have  taken  a whole  different  approach.  Maybe  it’s  like  the  comfort  level.  . . . I’m  more 
confident  now  than  I was  eight  years  ago.”  Having  an  individual  folder  for  each  teacher  with  student  assessment 
data.  Principal  Jones  explained  that  “the  [DW]  courses  are  making  the  impact  [on  me]  and  making  me  a better 
leader.  I’m  more  confident  of  myself.  I can  see  that  the  things  that  I’m  learning  from  here  and  that  they’re  helping 
the  school.”  Since  their  participation  in  the  DW  course.  Principal  Jones  notes  that,  in  meetings  with  individual 
teachers  as  well  as  in  departmental  meetings,  they  constantly  use  the  DW  products  in  order  to  “help  design  how  a 
teacher  is  addressing  the  children”  in  an  differentiated  way. 

As  shown  in  the  picture  below.  Principal  Jones  displays  a board  in  her  office  which  shows  every  teacher's  student 
assessment  data  as  color-coded  dots  for  individual  students  (e.g.,  red  dots  for  lowest  performing  vs.  green  dots  for 
high  performing  students)  so  that  everyone  can  easily  monitor  their  progress  over  time.  Thanks  to  the  school-wide 
approach  to  the  data,  every  teacher  and  administrator  at  CNE  understands  and  engages  with  each  other,  with  parents, 
and  with  the  NYC  quality  reviewers. 
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Picture:  Each  teacher’s  student  assessment  data  displayed  in  Principal  Jones’s  office] 


2.  TAM 

To  Principal  Ellis,  her  experience  in  the  Data  Wise  (DW)  course  proved  to  be  “very  effective  and  useful'’  and  helped 
her  develop  the  engagement  skills  she  needed.  She  wrote  in  her  survey  that  “It  [the  DW  course]  was  very  practical 
and  allowed  me  to  see  my  weakest  area  as  an  administrator:  engagement  of  the  whole  faculty.  In  addition,  it  gave 
me  a range  of  procedures  and  protocols  to  enhance  the  ability  to  involve  the  whole  faculty  in  many  different  ways.” 
[End-of-course  online  survey,  Spring08,  TAM-DW,  10411]  In  addition  to  establishing  school-wide  groups  focused 
on  student  assessment  data.  Principal  Ellis  improved  communication  with  her  staff  and  teachers  and  began  sharing 
her  decision-making  authority.  “I  ask  a lot  more  questions.  I have  been  allowing  the  teachers  to  take  more  ownership 
of  running  the  meetings.  Not  just  running  the  meetings  but  making  decisions.  That  has  really  made  our  teams  sort  of 
responsible  for  the  things  that  we  decide.  So  it  is  not  a decision  that  I have  made;  rather  it  is  a decision  we  have 
made,”  explained  Principal  Ellis. 


Reflective  Collaboration 


1.  CNE 

One  of  the  most  critical  ingredients  for  CNE’s  success  was  the  professional  collaboration,  working  together  as  the 
“Harvard  Team”,  every  Thursday  afternoon.  When  gathered  in  the  school  library,  the  faculty  could  work  in  close 
proximity  to  one  another. 

During  the  first-year  interviews  in  Year  1,  all  CNE  team  members  reported  that  they  benefited  from  this  established 
collaborative  time.  One  teacher  said  with  a sense  of  pride  at  their  accomplishment,  “It  was  a long  day  but  it  was  our 
‘Harvard  day’...” 

Another  teacher  also  described  Jones’s  never-ending  enthusiasm  and  her  full  participation  in  the  WIDE  courses. 
“She  [Jones]  led  by  example.  Ms  Jones  had  her  professional  development  done,  but  she  went  [to  the  DW  course] 
and  dug  into  that  data.  She  came  back  and  she  modeled  for  us.  She  said,  ‘this  is  what  I learnt’  and  therefore  we  are 
going  to  do  it  together.  So  it  was  a team  effort. 

Going  into  its  second  year,  CNE  participants  agreed  that  working  together  as  a team  “just  seems  to  go  by  faster.” 
According  to  one  teacher,  “we  have  been  on  the  same  team  and  now,  we  really  have  come  to  a point  where  we  can 
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sit  down  in  a lesson  and  immediately  figure  out  who  can  do  which  parts  [of  the  WIDE  assignments]  and  then  come 
back  together  and  say  “ok,  this  is  what  we  have  and  this  is  what  we’re  doing.”  [CNE  FG1]  In  an  online  survey,  one 
CNE  teacher  reported  that  “Being  able  to  work  with  team  members  was  fantastic.  I was  able  to  build  relationships 
with  co-workers  that  I previously  did  not  know  in  this  way.  It  was  great  to  learn  new  things  together,  bounce  ideas 
off  each  other,  and  have  professional  discussions  about  topics  that  we  were  all  interested  and  passionate  about.” 
[End-of-course  online  survey,  Fall08,  CNE-MI,  11585] 

This  increased  level  of  reflective  collaboration  among  teachers  increased  their  use  of  that  strategy  in  their 
classrooms.  “We  [teachers]  definitely  seem  to  be  more  ahead  than  we’ve  ever  been.  It  trickles  down  from  us  to  the 
way  we  instruct  the  kids,  because  you  put  them  in  a group  and  you  get  to  find  out  which  child  is  stronger  in 
something  and  which  one  can  do  something  else.  Then  they  all  work  together.  So  it’s  like  one  child  teaching  another 
child.  So  it’s  starting  from  us  and  going  down  to  them,”  explained  a teacher.  [CNE  FG2_TSNT1]  This  comment  was 
echoed  by  Principal  Jones,  “So  we’re  doing  [in  WIDE  courses]  what  we're  trying  to  get  the  kids  to  do.  . . . We’re  in  a 
collaborative  group,  and  we’re  taking  ownership  of  each  other  and  their  learning,  and  we  all  depend  on  each  other.” 
[CNE  Jones2] 

2.  ECE 

At  ECE,  where  13  teachers  took  the  Differentiating  Instructions  |DI]  course  as  a group,  the  participants  explained  “it 
builds  community.  We  have  many  teachers  from  different  departments:  from  special  education,  from  early 
education,  etc.  When  you  talk  about  a vertical  alignment,  that’s  what  happening  now.  This  course  was  a wonderful 
opportunity  for  collaboration  across  school,”  [ECE_FG1]  according  to  another  teacher,  “the  [DI]  course  has  forced 
us  to  have  cross-grade  conversations.  It’s  not  just  one  grade  but  we  have  to  work  with  different  grades  and  we  hear 
different  opinions  of  how  they  work  and  apply  what  we’re  learning  in  this  course  across  the  grades.”  [ECE_FG1] 
“This  course  was  an  excellent  experience  for  all  who  participated.  It  brought  us  closer  as  a professional  learning 
community,  and  established  protocols  for  collaborative  thinking,  and  professional  discussion  around  a topic  of 
importance  to  us  all.”  [End-of-course  online  survey,  Fall08,  ECE-DI,  10833] 

Principal  Miller  gave  a concrete  example  the  way  he  and  his  teachers  used  this  opportunity  to  meet  their  school's 
needs.  “I  specifically  collaborated  with  3rd  and  5th  grade  teachers  to  build  a coherent  program.  We  talked  with  3rd 
and  4th  grade  teachers  about  what  we  can  do  earlier  so  that,  when  kids  go  to  5th  grade,  they  are  as  capable  as 
possible.  We’ve  been  just  talking  about  how  we  do  what  we  need  to  do  and  still  present  the  material  in  a way  that  is 
accessible  for  5th  graders.  This  course  made  several  5th  grade  teachers  collaborate  and  think  together.  It  is  really 
wonderful  to  see  the  seamlessness  going  on  through  the  differentiating  instruction,”  explained  Principal  Miller. 

Principal  Miller  explained  that  “the  difference  between  a good  school  and  a great  school  is  that  a good  school 
teaches  and  a great  school  teaches  and  learns.”  His  emphasis  on  life-long  learning  was  particularly  evident  at  ECE. 
The  “Harvard  Team”  was  “really  about  learning  - learning  at  work.”  Principal  Miller  explained,  “I  tell  them 
[teachers]  that  I know  I have  tried  a thousand  things  and  failed  at  a lot  of  things.  So  I give  them  a lot  of  opportunities 
to  learn  from  that  experience  because  that  is  the  way  I learned  ...  in  order  to  make  teachers  feel  very  comfortable 
and  very  supported.  So  they  can  come  and  talk  about  their  failures  and  lesson  learned  from  that  experience.  They 
sense  that  trust.” 

3.  TAM 

Collaboration  similar  to  that  at  ECE  took  place  at  TAM,  in  the  second  year.  “The  team  members'  role  for  each 
session  was  given  ahead  of  time,  so  doing  the  task  for  a particular  session  was  just  easy,”  one  teacher  reported  in  the 
end-of-course  online  survey  [Spring08,  TAM-DI,  10388]  Another  teacher  in  the  same  course  highlighted  the  mutual 
support  that  each  team  member  gave  and  received:  “my  team  work  was  quite  helpful  to  me  because  I am  not  too 
conversant  with  navigating  information  from  one  [internet]  site  to  another.  But  my  fellow  teammate  is  computer 
literate,  so  we  were  able  to  navigate  our  postings  and  links  from  one  site  to  the  other  very  easily.”  [End-of-course 
online  survey,  Spring08,  TAM-DI,  10390] 

To  meet  the  internal  needs  for  improved  communication  and  collaboration,  TAM  teachers  came  up  with  seven 
cross-grade  and  cross-departmental  Inquiry  Teams  to  help  out  lowest  performing  students  in  the  school.  In  the 
2007-08  year,  TAM  had  only  one  Inquiry  Team.  Thanks  to  their  successful  results,  they  have  added  six  more 
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Inquiry  Teams  in  2008-09  that  will  target  a student  group  with  the  lowest  academic  scores  in  math  exams.  This 
teacher  explains  the  process  in  detail,  “last  year,  we  had  one  Inquiry  Team  to  for  the  whole  school  and  we  followed 
the  same  idea  [this  year]  as  we  discussed  how  to  run  it,  how  to  find  better  practices,  how  to  build  an  action  plan  on 
that.  . . . We  have  seven  [Inquiry]  Teams  now.  And  we  are  focusing  on  any  student  that  scored  a 1 [the  lowest  score] 
on  their  math  [state  exam].  We  are  trying  to  figure  out  how  to  help  those  forty-one  l’s  and  trying  to  find 
interventions  to  help  them.”  [TAM  FG1]  According  to  Principal  Ellis,  “now  we  made  up  enough  [Inquiry]  teams  to 
make  sure  that  they  are  school-wide.  We  have  the  over-age;  we  have  bilingual;  we  have  the  ELA;  we  have  math.  I 
mean  we  have  every  need  that  you  can  think  of.  And  we  have  [data]  matrix  to  show  what  they  need,  so  we’re 
watching  them.”  [TAM  Ellis] 

V.  Outcomes  for  Students 

In  this  section,  we  describe  indirect  impacts  of  teacher  and  leader  improved  professional  practice  on  student 
performance.  Since  WIDE  did  not  have  direct  interaction  with  students,  the  following  examples  were  selected  from 
interviews,  teaching  portfolios,  and  open-ended  questions  in  our  end-of-course  online  surveys  completed  by  the 
participating  teachers  and  leaders. 

Overall,  each  portfolio  contained  a series  of  comprehensive  lesson  plans  with  goals,  relevant  NYC  state  standards, 
sequence  of  different  performances  (e.g.,  independent  or  group  work),  materials/equipment  to  be  used,  and,  in  some 
cases,  student  homework.  The  teachers’  Reflection  Logs  and  the  classroom  photographs  show  teachers  and  students 
engaged  in  these  lessons. 

For  instance,  Mr.  Scott,  a 4th  grade  ECE  teacher  at  ECE,  applied  the  TfU  framework  to  organize  and  document  his 
environment  unit.  His  portfolio  clearly  showed  how  his  students’  learning  activities  were  related  to  specific 
understanding  goals  and  different  performances  throughout  the  unit.  The  two  portfolios  from  TAM,  in  the  subjects 
of  English  and  science,  were  mostly  a collection  of  teaching  materials  and  the  students’  worksheets.  This  might  be 
due  to  the  fact  that  middle  grade  work  is  more  worksheet-based  than  is  elementary  school  work.  Another  difference 
between  ECE  and  TAM  portfolios  was  that  TAM  portfolios  contained  mostly  short,  hand-written  comments  on 
student  work  without  elaborated  explanations  as  to  how  their  rubrics,  included  in  the  portfolios,  were  used  to  assess 
the  work. 

Despite  these  limitations,  the  portfolios  show  a rich  and  diverse  collection  of  materials  from  the  teachers  who 
generously  shared  them  with  us.  All  portfolios  showed  various  levels  of  improvement  in  terms  of  their  students’ 
understanding,  engagement,  and  reflective  collaboration  and  provide  details  that  highlight  those  three  SEIM  model 
categories  for  student  outcomes. 

Understanding  for  students:  Developing  and  demonstrating  mastery  of  the  learning  goals,  including  an 
increased  capability  to  apply  knowledge  creatively  and  appropriately  in  a range  of  circumstances,  which 
are  consistent  with  the  TfU  framework. 

Engagement  of  students:  The  sense  of  student  enthusiasm,  interest  in,  commitment  to,  and  caring  about 
their  learning  and  school  work,  including  intrinsic  or  self-generated  motivation. 

Reflective  collaboration  for  students:  The  capability  and  desire  to  participate  in  inquiry  that  includes 
working  together  effectively  with  one's  peers  towards  common  learning  goals. 


Understanding 

Two  CNE  teachers  took  the  Multiple  Intelligences  [MI]  course  as  a team  in  the  Fall08  term  and  explained  how  the 
course  changed  their  understanding  of  student  academic  performance  and  self-esteem.  One  of  the  CNE  teachers 
explained,  “I  am  a teacher  working  with  economically  disadvantaged  youth  in  an  inner  city  school.  We  are  trying  to 
help  students  take  ownership  of  their  learning  by  making  them  aware  of  their  [academic]  levels,  their  strengths  and 
their  weaknesses.  Unfortunately,  because  we  as  teachers  are  always  looking  to  see  what  else  a student  needs  to  learn, 
there  tends  to  be  a focus  on  student  weaknesses.  This  can  lead  to  low  self-esteem  in  the  students.  Through  the  MI 
course,  I am  starting  to  change  my  lens  of  focus.  Instead  of  focusing  on  weaknesses,  the  Multiple  Intelligence  theory 
looks  at  student  strengths  and  intelligences.  The  student  is  validated  for  what  they  can  do  and  the  teacher  sees  the 
strengths  as  entry  points  to  move  towards  supporting  the  weaknesses.  We  are  now,  as  a professional  development 
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team,  bringing  this  focus  to  the  teachers  in  our  school.”  [End-of-course  online  survey,  Fall08,  CNE-MI,  1 1585]  A 
second  teacher  added,  “Our  school  has  always  struggled  with  the  challenges  that  present  themselves  with  regards  to 
student  performance  and  motivation.  We  have  looked  at  student  weaknesses  and  tried  to  address  them  without 
looking  at  students'  strengths  and  intelligences  as  inroads  to  deal  with  those  deficits.  Through  this  [MI]  course,  I 
have  had  a complete  change  of  focus.  I now  look  at  students  through  different  lenses.  I try  to  discover  students' 
strengths  and  intelligences  and  use  them  to  help  them  better  and  possibly  master  their  weak  areas.  . . . Students  are 
becoming  more  motivated  because  we,  as  teachers,  are  using  the  information  to  reach  the  students  and  we  are  also 
using  real-life  connections  to  help  see  how  what  they  learn  is  relevant  to  their  lives.”  [End-of-course  online  survey, 
Fall08,  CNE-MI,  1 1582] 

An  ECE  teacher  explained  ways  in  which  he  applied  Differentiating  Instruction  to  his  learning  activities  and  student 
assessments  in  order  to  improve  student  understanding.  In  the  middle  of  teaching  an  election  unit,  he  decided  to 
“modify  certain  aspects  [of  learning  activities]  and  broaden  the  scope  to  allow  more  methods  of  production.”  His 
expanded  lesson  plan  included  “music,  map  making,  mock  elections  and  more  as  activities.  Within  those  activities, 
students  were  able  to  create  alternate  forms  of  product  for  assessment.”  He  continued  that  “having  one  mode  of 
assessment,  in  my  estimate,  could  only  benefit  about  40%  of  the  class,  the  middle.  It  neglects  both  the  30%  below 
and  the  30%  above.  Having  created  alternate  ways  for  students  to  show  their  knowledge  allowed  for  a more  robust 
and  exciting  teaching  environment.  In  addition,  the  students  were  able  to  take  away  more  from  the  lessons.”  As  a 
result,  he  indicated  that  he  reached  “more  students  with  a deeper  meaning  of  the  election"  and  realized  more 
individual  “talents  and  capabilities”  that  he  can  integrate  into  other  subjects  in  the  later  units.  [End-of-course  online 
survey,  Fall08,  ECE-DI,  15055] 

For  the  middle  school  students  at  TAM,  using  various  technologies  and  small  group  work  helped  them  to  focus 
on  their  learning  and  improve  their  in-class  behavior.  In  her  end-of-course  online  survey,  one  teacher 
explained,  “Initially,  I found  myself  explaining  the  skills  to  students  repeatedly.  The  idea  was  the  use  of 
technology  that  simulates  the  skills.  Students  followed  a model  and  worked  independently  with  technology  to 
solve  and  graph  from  simple  to  more  complex  inequalities.  The  Teaching  for  Understanding  [framework] 
through  the  use  of  appropriate  and  effective  technology  enhanced  students  understanding.  Once  they 
understood,  they  did  not  want  to  stop  working”.  [End-of-course  online  survey,  Fall07,  TAM-TSNT1,  10394] 

Another  teacher  reported  in  her  survey,  “In  my  class,  I have  25  students  who  learned  at  different  pace  and 
some  with  behavior  problems.  The  use  of  differentiated  instruction  really  helped.  Even  the  most  challenged 
students  achieved  learning  with  understanding  at  their  own  pace.  If  I hadn't  used  the  idea  of  differentiated 
instruction,  the  most  challenged  students  would  find  it  difficult  to  catch  up  with  the  more  able  group.  Hence, 
there  would  be  less  motivation  and  some  behavior  problems.  Students  are  motivated  to  do  the  task  and  they 
love  the  idea  that  they  can  present  concepts  in  line  with  their  interest  as  well  as  do  self-evaluation  and  peer 
critique.”  [End-of-course  online  survey,  Spring08,  TAM-DI,  10388] 

A further  example  from  the  end-of-course  survey,  “I  work  with  students  who  have  diverse  needs.  During  any 
class  activity,  the  students  work  in  groups  of  3-4  students  who  are  made  up  of  general  education  and  special 
education  students.  I differentiated  the  task  according  to  the  students'  abilities.  The  students  demonstrated  their 
understanding  of  the  writing  assignment  differently  while  some  students  chose  their  own  topic  to  write  on.  The 
other  group  was  given  a topic  with  samplers  and  graphic  organizers  to  help  them  gather  their  ideas  in  a format. 

At  the  end  of  the  assignment,  both  groups  of  students  completed  their  task  appropriately.  Differentiating  the 
task  enabled  each  group  of  students  to  demonstrate  their  knowledge  of  writing”.  [End-of-course  online  survey, 
Spring08,  TAM-DI,  10390] 

A TAM  8th  grade  English  teacher  described  the  way  she  implemented  the  WIDE  course  ideas  with  her  students,  who 
are  mostly  English  Language  Learners  (ELL).  In  her  adolescent  literature  unit  reading  ‘The  Watsons  Go  to 
Birmingham”,  the  class  worked  to  identify  issues  related  to  the  history  of  school  segregation  and  the  U.S.  civil  rights 
movement.  Her  differentiating  approaches  to  the  unit  included: 

1)  Student  readings  were  differentiated  by  interest  and  ability  level  and  similar  level  students  worked  together  as  a 
small  group 

2)  Students  studied  texts  using  a variety  of  media  (i.e.,  picture  books,  images,  and  graphic  organizers)  to 
communicate  their  understanding 
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3)  Students  were  given  ample  opportunities  to  assess  their  own  learning  using  a variety  of  self-assessment  tools. 

Throughout  the  unit,  this  teacher  provided  the  Learning  Log  Questions  and  what  she  called  “Ticket  out  the  Door 
Questions”.  After  the  lesson,  each  student  had  to  answer  these  and,  in  doing  so,  reflected  on  their  own  learning.  In 
her  Class  Reflection  logs,  this  instructor  indicated  that  most  of  her  lesson  goals  were  met  and  her  students  “formed  a 
broad  understanding  of  the  texts  while  determining  the  important  and  relevant  information.”  She  also  observed  an 
increased  level  of  engagement,  noting  that  the  students  remained  on  task  during  the  lesson  and  sometimes  stayed 
after  to  class  to  continue  the  discussion 

Engagement 

At  CNE,  a first-grade  teacher  described  student  engagement  in  her  class,  “What  we  [her  class]  can  learn,  what’s 
important  to  them,  and  how  we  can  fix  a problem  [of  the  unit].”  Asking  her  students  for  their  input  on  lessons 
proved  to  be  very  helpful  for  her  as  she  structured  her  learning  activities.  “I  do  use  the  input  of  the  children  more, 
and  actually  give  them  the  problem  of  saying  ‘how  are  we  going  to  fix  it,  what  are  we  going  to  do.’  This  is  working 
for  me  and  even  just  in  the  first  grade,  they’re  pretty  good  at  it,”  explained  the  teacher.  [CNE  FG2_TSNT1] 

Using  student  data  in  the  classroom  also  improved  the  level  of  student  engagement  as  they  assessed  their  own 
progress  at  CNE.  There  were  a number  of  teachers  who  shared  exciting  examples.  One  teacher  explained,  “I 
created  a graph,  so  that  the  kids  could  look  and  see  [their  assessment  data].  ‘Wow,  ok,  I started  here,  and  then  I 
move  this  much  from  September  to  January  and  then  I moved  that  much  from  January  to  March  and  this  much 
from  March  to  June.'  So  they  can  see  themselves,  what  they  did.”  A second  teacher  added,  “The  children  had 
developed  goals  based  on  that  data.  So  the  children  know,  ‘I’m  right  here  now,  I’m  a 2 (in  the  assessment 
rating).  ...  They  know  the  terms,  and  they  know  what  they  need  to  do  to  move  up  there.  So  our  teachers  have 
really  bought  into  it  and  they  have  channeled  that  information  to  the  children,  and  they  do  that  to  the  parents  as 
well.”  [CNE  FG1_MI]  Another  teacher  shared,  “after  a while  [sharing  the  student  data],  the  children  realized 
the  importance  of  the  movement  rather  than  the  score.  It  was  how  much  gains  I made,  and  then  that  was  the 
motivating  factor.  It  wasn’t  that  I was,  maybe  a bottom,  but  look  at  the  gains  I made,  and  that  made  the 
difference.”  [CNE  FG1_MI] 

One  ECE  teacher  shared  his  story  about  how  his  scaffolding  effort  in  small  groups  helped  to  engage  students  in 
the  lessons.  “I  was  trying  to  scaffold  my  students'  understanding  of  division  and  its  real  life  applications.  I 
decided  to  group  my  students  heterogeneously,  and  to  have  that  group  simulate  ordering  a Chinese  food  meal 
consisting  of  an  appetizer,  a soup,  and  an  entree.  Groups  were  provided  with  hard  copy  menus  for  choosing 
their  items,  and,  respectively,  decide  the  breakdown  of  monetary  contributions  by  each  member  to  pay  for  the 
order.  Discussions  about  math  took  place  and  students  learned  about  ‘consensus.’  Students  designed  charts  and 
displayed  their  work.  I am  now  more  focused  on  designing  activities  requiring  more  group  work,  especially 
during  mathematics.  This  was  one  of  the  most  engaging  lessons  for  my  students...  they  continued  to  work  a 
little  into  their  recess  period.”  [End-of-course  online  survey,  Fall08,  ECE-DI,  15052] 

At  TAM,  integrating  technologies  into  various  curricula  activities  greatly  improved  student  engagement  and 
many  teachers  shared  their  stories  about  incorporating  a range  of  WIDE  course  ideas  into  their  lessons.  One 
teacher  shared  this  insight,  “The  idea  I got  from  this  [TSNT1]  course  is  the  several  web  sites  and  technology 
resources  that  were  useful  in  our  lesson  design.  I added  them  as  part  of  the  lesson  design.  This  technology  tool 
makes  the  lesson  more  interactive  and  encourages  most  challenged  students  to  participate  fully  in  the  lesson.  In 
my  class,  there  are  many  struggling  readers  but  with  the  use  of  technological  tools  and  resources  I have 
acquired  from  this  course,  these  students  will  be  able  to  participate  and  interact  with  their  groups.”  [End-of- 
course  online  survey,  Fall08,  TAM-TSNT1,  10390] 

Two  8th  grade  science  teachers  from  TAM  created  a unit  on  understanding  mass,  volume  and  density  based  on 
the  principles  learned  in  their  TSNT1  and  DI  courses.  In  the  new  unit,  students  first  learned  how  to  measure 
and  record  the  mass  and  volume  of  objects  using  scientific  instruments.  Then  they  explored  the  concept  of 
density  through  hands-on  experimentation  and  group  discussions.  Lastly,  they  demonstrated  their 
understanding  of  these  concepts  through  individual  work  and  summary  reports.  The  teachers  developed  new 
rubrics  for  student  work  that  reflected  the  variety  of  student  performances.  In  the  Class  Reflection  Log,  the 
teachers  indicated  that  students  were  “very  creative”  and  showed  “intrinsic  motivation”  and  “ownership  of 
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their  learning”  because  they  were  involved  in  the  rubric  development,  conducted  hands-on  experiments,  and 
collaborated  with  one  another  in  their  small  groups.  They  also  allowed  students  to  produce  differentiated 
products  based  on  their  learning  styles.  As  a result,  the  teachers  reported  that  “our  students  enjoyed  what  they 
were  doing  and  their  performance  was  of  high  standard,  based  on  the  rubrics.” 

Reflective  Collaboration 

Mr.  Scott  is  a 4th  grade  teacher  at  ECE  who  took  Differentiating  Instructions  (Fall  2008)  and  Teaching  for 
Understanding  1 (Spring  2009),  submitted  our  third  portfolio.  His  class,  comprised  of  25  students  from  many 
different  parts  of  the  world,  including  Latin  America  and  the  Caribbean,  had  varying  levels  of  English  reading 
abilities.  His  portfolio  unit  was  an  interdisciplinary  project  on  the  environment,  central  to  4th  grade  social  studies, 
science  and  literacy.  Mr.  Scott  applied  the  DI  strategies  with  the  TfU  framework  to  design  and  assess  this  unit. 

The  project’s  overarching  goal  was  to  increase  student  awareness  of  the  environment  around  them  and  to  understand 
their  role  in  its  change.  Students  also  turned  their  research  into  action  and  created  a variety  of  projects  that  made 
their  learning  tangible. 

His  four  understanding  goals  (UGs)  were: 

1.  The  environment  is  all  living  and  non-living  things  on  earth.  Keeping  it  healthy  is  vital  for  all. 

2.  Humans  have  the  greatest  effect  on  the  environment  and  can  make  a huge  impact  on  its  health. 

3 . Changes  in  one  part  of  the  environment  and  ecosystem  have  effects  on  other  parts  of  the  environment  and 
ecosystem. 

4.  By  conserving  natural  resources  and  practicing  the  3R’s  (recycle,  reduce,  and  reuse)  we  can  keep  our  planet 
healthy. 

In  order  to  achieve  these  goals,  Mr.  Scott  designed  a series  of  Performances  of  Understanding  which  included  two 
initial  performances,  one  inquiry-based  performance,  and  two  culminating  performances,  and  connected  the 
performances  to  the  understanding  goals. 

The  following  descriptions  provide  a brief  summary  of  each  performance  and  some  selected  part  of  Mr.  Scott’s 
reflections  from  his  Reflection  Logs. 

• Initial  Performance  1:  Recycling  Program  in  classroom  (addressing  UGs  2 & 3) 

As  an  introduction  to  the  unit,  Mr.  Scott  brought  in  a range  of  everyday  objects.  After  discussing  different  materials 
we  discard,  students  broke  into  groups  and  created  definitive  lists  to  separate  the  items  into  categories  (e.g.,  plastic, 
paper,  trash).  Based  on  this  small  group  activity,  the  class  as  a whole  created  a recycling  program  in  which  all 
students  participated. 

Mr.  Scott’s  reflection  on  the  recycling  program  reported  that  “students  in  similar  reading  groups  were  able  to 
collaborate  better.  I realized  that,  if  I put  my  students  in  groups  based  upon  reading  level,  I could  give  them 
appropriate  reading  materials.” 

• Initial  Performance  2:  100  Day  Challenge  (addressing  UGs  1,  3,  & 4) 

The  second  initial  performance  happened  on  the  100th  day  of  school.  Students  got  into  groups  and  discussed  ways  in 
which  they  can  help  the  environment.  After  a discussion  period,  students  select  categories  to  act  upon  such  as 
recycle,  conserve,  and  so  on.  Each  group  then  took  responsibility  to  come  up  with  20  ways  to  help  the  environment 
and  displayed  their  ideas  in  a planet  poster  on  the  classroom  wall.  A representative  student  for  each  group  made  a 
presentation  to  the  class. 

In  his  reflection  log  on  this  performance,  Mr.  Scott  indicated  that  “there  was  great  excitement  in  trying  to  reach  the 
100th  day  mark  of  the  school.  The  students  felt  personally  responsible  for  being  the  ones  to  tell  the  rest  of  the  school 
how  to  help  the  planet.  It  was  95%  success  with  great  ideas  and  group  work.  The  project  came  together.” 

• Inquiry  Performance:  Environmental  Booklet  (addressing  UGs  2 & 3) 
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This  performance  consisted  of  several  written  assignments.  Students  first  prepared  a non-fiction  writing  piece  by 
examining  what  the  environment  is  and  why  it  is  important  to  us.  Students  then  collected  pictures  from  magazines 
and  periodicals  and  designed  an  environmental  booklet  as  a group  to  inform  the  public  about  their  impact  on  the 
environment. 

“The  process  was  clear  and  the  student  involvement  was  high.  Students  showed  materials  and  ideas  while  creating 
their  piece.  Students  with  higher  creativity  were  paired  with  those  less  with  success.  The  booklets  were  well  done 
and  students  spent  a lot  of  time  and  didn't  rush.  Creativity  and  interests  were  high,”  wrote  Mr.  Scott.  By  pairing  up 
students  with  mixed  abilities,  he  attempted  a different  mode  of  collaboration  among  students. 

• Culminating  Performance:  Formal  Assessment  (addressing  UGs  1,  2,  3,  & 4) 

Mr.  Scott  was  also  aware  of  the  needs  for  the  formal  assessment  of  his  unit.  Based  on  the  materials  in  his  portfolio, 
he  used  multiple-choice  and  open-ended  questions  to  assess  his  students’  understanding  of  the  topics  with  regard  to 
all  four  understanding  goals  of  the  unit. 

• Culminating  Performance:  Final  Project  (addressing  UGs  1,  2,  3,  & 4) 

As  a last  culminating  performance,  Mr.  Scott  assigned  students  to  groups,  depending  on  their  learning  style.  Within 
their  groups,  students  discussed  what  they  learned  throughout  the  unit  and  demonstrated  their  understanding  in  one 
of  the  following  performances:  a dramatization,  a poster,  a PowerPoint  presentation,  a pamphlet,  or  a toy  made  from 
recycled  goods.  At  the  last  lesson  of  the  unit,  students  performed  the  dramatizations,  passed  out  their  pamphlets  to 
other  students  in  the  school,  displayed  their  posters,  and  shared  their  recycled  toys  with  the  lower  grades. 

Mr.  Scott  pointed  out  some  difficulties  in  the  approach.  “My  pairing  of  students  did  not  always  mean  that  the  project 
I selected  was  perfect  for  them.  Some  students  were  not  getting  along  though  they  had  the  same  learning  styles,” 
confessed  Mr.  Scott.  Despite  this,  he  explained  that  his  students’  interest  remained  high  throughout  the  unit.  Fie 
realized  that  “having  a variety  of  performances  centered  on  the  students’  best  skills  had  the  students  feel  like  they 
were  prioritized.”  Mr.  Scott’s  portfolio  clearly  showed,  his  WIDE  participation  helped  him  develop  and  attempt  a 
variety  of  opportunities  for  his  students  to  collaborate  in  their  learning  activities. 

VI.  Systemic  Shifts 

In  addition  to  the  individual-level  educational  improvements,  our  data  showed  systemic  improvements  at  these 
schools,  as  a result  of  their  experience  with  WIDE,  in  the  areas  of  Educational  Culture,  Educational  Standards  and 
Tools,  Organizational  Structures,  and  Visionary  and  Distributed  Leadership.  As  emphasized  in  the  beginning  of  this 
paper,  these  elements  interact  with  one  another  in  an  ongoing  and  cyclical  way.  Improved  systemic  conditions,  for 
example,  not  only  change  the  existing  context  for  more  effective  planning  and  implementation,  but  also  create  a new 
context  for  further  levels  or  forms  of  educational  improvements. 

Keeping  in  mind  the  NYC  public  schools  accountability  requirements,  the  following  analysis  considers  ways  in 
which  these  schools'  teachers  and  leaders  used  the  WIDE  program  to  effectively  achieve  their  educational  priorities 
for  the  last  two  years  as  well  as  other  relevant  initiatives  that  happened  before  and  during  the  WIDE  participation. 

Educational  Culture 

In  the  NYC  schools  studied  in  Year  1,  we  observed  various  changes  in  school  culture  due  to  the  WIDE  PD 
activities.  For  example,  creation  of  clear  and  common  goals,  the  development  of  shared  language  to  practice  and 
assess  effective  teaching  and  learning,  and  more  open  and  supportive  interactions  and  collaborations  among  teachers 
and  administrators.  Some  of  the  participants  in  Year  1 identified  the  WIDE  contribution  to  the  development  of 
“open-mindedness,”  becoming  “reflective”  practitioners,  “taking  risks,”  asking  critical  questions,  and  attuning  to 
“lifelong  learning.” 

In  Year  2,  the  course  participants  in  the  three  schools  explained  how  their  WIDE  participation  supported  positive 
changes  in  their  school  culture.  Many  of  these  changes  were  detailed  in  previous  sections  of  this  report,  but  the 
description  of  educational  culture  transformation  at  TAM  is  worth  highlighting. 
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Principal  Ellis,  at  TAM,  explains  what  it  was  like  when  the  school  first  opened.  “In  the  beginning,  there  was  a lot  of 
animosity...  among  the  teachers  and  towards  the  leadership  team.  Teachers  were  more  passive,”  recalled  Principal 
Ellis.  [TAM  Ellis  1 Feb09]  When  the  leadership  team  called  for  a staff  meeting,  for  example,  it  often  became  “a 
place  to  vent  out  [teachers'  frustration  or  dissatisfaction].  We  were  there  to  vent.”  Therefore,  these  meetings  were 
used  “to  generally  grumble  or  complain  about  specific  students”  in  the  beginning  of  her  leadership. 

After  taking  WIDE’s  Leading  for  Understanding  (LfU)  course.  Principal  Ellis  and  her  leadership  team  realized  the 
school-wide  changes  they  had  to  make.  They  knew  they  needed  to  create  a common  vision  of  TAM  as  “an  open, 
calm,  and  welcoming  place  for  students,  parents  and  teachers  alike.”  [TAM  Ellis  1 Feb09]  In  order  to  do  that,  the 
first  step  was  to  make  this  vision  public.  “You  are  going  to  see  the  mission  and  vision  of  the  school  everywhere  [in 
the  school],  said  Principal  Ellis.  [TAM  Ellis  1 Feb0409]  At  the  same  time,  the  leadership  team  encouraged  the 
teachers  to  participate  and  to  be  more  engaged  in  the  decision-making.  For  example.  Principal  Ellis  now  meets  one- 
on-one  with  each  teacher  three  times  a year. 

Compared  to  our  first  visit  in  the  fall  of  2007,  we  saw  significant  improvement  in  the  school  environment  in  the 
second  year.  There  was  classical  music  playing  in  the  hallways  and  teachers  and  students  were  seen  exchanging 
warm  greetings.  In  the  interviews  with  Principal  Ellis  and  the  WIDE  participating  teachers,  the  term  “nurturing”  and 
“caring”  was  used  to  describe  the  school  environment. 

Thanks  to  the  improved  understanding  and  engagement  among  teachers  and  leaders,  their  meetings  are  “different” 
and  the  TAM  teachers  now  take  more  “ownership,”  according  to  Principal  Ellis.  “That  was  very  rough,  but  we  were 
able  to  surpass.  I was  able  to  present  it  [the  change]  to  them  to  show  that  it  would  benefit  all  of  us.”  [TAM  Ellis  1 
Feb09]  “Now,  they  [teachers]  talk  about  what  can  be  done  to  improve  things  in  the  school. . . The  conversation  is 
wonderful,”  said  Principal  Ellis.  [TAM  Ellis  Nov20] 

Curriculum,  Standards,  & Tools 

To  make  school-wide  systemic  changes,  the  three  schools  undertook  critical  examinations  of  their  existing  curricula, 
standards,  and  assessment  tools  used  to  meet  the  NYC’s  accountability  requirements. 

In  the  Fall07  term,  the  CNE  Leadership  Team  divided  into  two  groups,  the  Quality  Review  Team  and  the  Inquiry 
Team,  while  taking  the  Leading  for  Understanding  1 (LfUl)  course.  One  teacher  in  the  Leadership  Team  explained 
how  these  two  groups  worked  together  to  develop  a leadership  capacity  in  the  school,  to  address  their  lack  of 
understanding,  and  to  make  better  use  of  student  assessment  data  in  order  to  improve  their  status  in  the  upcoming 
Quality  Review. 

The  two  groups  became  active  members  of  our  leadership  team.  ...  “We  all  share  the  common  vision  that  our  school 
and  staff  provide  students  with  the  necessary  tools  to  be  lifelong  learners.  First,  we  used  this  [LfU]  course  to  prepare 
for  our  annual  Quality  Review  which  rates  the  school  on  how  we  use  data  to  drive  instruction  and  promote  student 
achievement.  Second,  gathering  in  an  after-work  situation  strengthened  our  relationships  and  respect  for  each  other. 
We  are  spending  more  time  working  collaboratively,  looking  at  new  ideas,  data,  programs,  and  ways  to  provide  our 
students  to  become  independent  learners  and  be  responsible  for  their  learning.”  [End-of-course  online  survey, 

Fall07,  CNE-LfU,  1 1583]  The  same  teacher  took  the  Data  Wise  (DW ) course  in  the  Spring08  term  and  reported  that 
“we  [the  leadership  team]  have  become  more  open  to  looking  at  all  aspects  of  our  teaching  - using  learning  walks 
and  student  talks  - meeting  as  a team  with  each  teacher  every  six  months  to  discuss  their  students.  Teachers  are 
becoming  more  effective  each  day  using  data  to  drive  lesson  planning  and  implementing  lessons.  They  are  seeing 
the  effectiveness  of  using  data.  We  are  seeing  positive  feedback  regarding  student  making  progress.”  [End-of-course 
online  survey,  Spring08,  CNE-DW,  11583] 

With  ongoing  help  from  the  Data  Specialist  in  the  school,  many  CNE  teachers  were  able  to  improve  their 
understanding  of  individual  student  assessment  data  as  well  as  create  their  own  graphs  and  charts  to  analyze  their 
students,  showing  progress  in  reading  or  mathematics  over  time.  During  the  third  term  (Fall08)  with  WIDE,  when 
the  whole  school  was  applying  the  DW  principles  to  their  data  use  and  analysis,  several  teachers  illustrated  concrete 
examples  of  the  way  they  integrated  DW  into  ideas  from  other  WIDE  course  taken  in  previous  terms.  Having  gone 
through  two  years  of  WIDE  courses,  one  teacher  at  CNE  summarized  their  improved  use  of  data  like  this,  “right 
now,  our  teachers  are  very  very  savvy  [with  the  data].  They  want  to  see  the  data.  They  want  to  use  it.  So  we  are 
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grouping  our  children.  We’re  looking  at  the  disaggregated  data,  how  well  our  students  are  doing,  the  ethnic  groups 
of  students,  and  how  the  boys  do  versus  the  girls.  ...  So  we’re  looking  at  the  data  from  every  aspect  possible  and  the 
teachers  are  buying  into  it  because  they  see  the  value.”  [CNE  FG1_MI] 

Thanks  to  the  longer  engagement  with  WIDE  courses,  ideas  from  different  WIDE  courses  have  been  integrated  into 
more  coherent  standards  on  effective  teaching  and  learning  at  CNE.  One  teacher  who  took  both  DataWise  and  the 
Multiple  Intelligences  (MI)  courses  said  that  “I  think  also  the  whole  approach  of  looking  at  the  performances  [data] 
and  different  types  of  [MI]  assessments,  kind  of  pulling  it  all  together.  I think  that  is  also  in  line  with  what  the 
school  is  trying  to  do:  looking  at  the  children  as  individual  learners,  looking  at  the  data  that  we  have  on  those 
children,  trying  to  figure  out  where  we  can  help  them  best,  and  where  we  are  weak,  and  where  we  need  to  get  better. 
And  I think  that  it  kind  of  all  pulls  together  in  the  same  spot.”  [CNE  FG2_TSNT1] 

At  ECE,  one  administrator  articulated  how  his  school’s  WIDE  participation  affected  every  aspect  of  the  learning  and 
teaching,  “Differentiation  is  an  essential  concept  in  both  the  elementary  school  classroom  and  when  providing 
professional  development  [PD]  to  a staff.  The  concepts  surrounding  differentiation  for  teachers  must  also  take  into 
consideration  the  entry  point  of  staff,  their  learning  styles,  content  of  core  strategies  and  product.  Teachers  are  eager 
to  help  one  another,  and  to  seek  support  that  addresses  their  individual  needs.  Teachers  are  now  focused  on 
designing  individual  goals  for  students,  and  students  are  part  of  the  process.  Students  are  now  able  to  describe  their 
own  learning  styles,  their  interests,  and  what  they  need  to  do  to  move  on  to  the  next  level.”  [End-of-course  online 
survey,  Fall08,  ECE-DI,  10833] 

CNE  also  carried  out  their  Quality  Review  preparation  based  on  their  learning  experience  in  the  Data  Wise  course. 

In  the  Spring08  term,  for  instance,  one  Inquiry  Team  participated  in  the  Data  Wise  (DW)  course  in  order  to  “decide 
what  area  to  work  on  with  our  chosen  [targeted  lowest  performing]  students”  and  combined  already  existing 
assessment  tools  in  the  school  while  taking  the  DW  course.  “The  Questioning  protocol  helped  us  [the  DW  team] 
brainstorm  to  choose  the  area  of  concentration.  We  then  displayed  a plan  and  assigned  team  members  for  different 
tasks.  We  created  a newsletter  to  share  results  with  the  staff  [in  the  school],”  reported  a member  of  this  Inquiry 
Team  in  the  course  survey.  [End-of-course  online  survey,  Fall07,  CNE-LfU,  10390] 

When  asked  to  reflect  upon  their  three-term  experience  with  WIDE  courses.  Principal  Ellis  described  the  impact  of 
various  WIDE  courses  in  the  following  way:  “The  first  [Teaching  to  Standards  with  New  Technologies  course] 
guided  us  a great  deal  and  improved  technology  in  our  school.  Even  though  the  name  [of  our  school]  is  the  School 
of  Science  and  Technology,  we  were  not  living  up  to  our  names  because  technology  was  not  really  in  [right]  places. 

. . . We  started  looking  at  data  critically  after  we  took  the  DW  course.  . . . When  it  comes  to  data,  we  used  to  have 
tons  of  data.  But  we  were  not  organized.  We  didn’t  even  know  why  we  collected  the  data.  Today,  we  have  that  data 
in  one  place,  we  know  why  we  have  it,  who  needs  it,  why  we  are  using  it,  how  we  can  benefit  from  the  data.  That 
data  is  utilized  to  communicate  with  parents.  That  data  is  utilized  to  have  conferences  with  the  students. . .for 
teachers  to  see  themselves.  That  data  is  embedded  in  the  goal  setting  at  the  beginning  of  the  school  year.”  [TAM 
Ellis  Non  20 1 

She  then  showed  a thick  binder  what  she  referred  to  as  “my  data  binder.”  “Every  single  teacher  has  their  own  data 
binder.  This  is  the  data  about  the  last  progress  report. . .We  have,  for  example,  [data]  for  English  Language  Arts. . . 
We  took  out  all  the  different  skills  that  were  assessed  in  the  tests.  Now,  the  teachers  will  be  able  to  see  that,  skill  by 
skill,  which  one  is  the  one  [to  work  on].  They  will  be  able  to  modify  and  have  differentiated  instructions  that  we 
learned  in  the  DI  course.  [TAM  Ellis  Nov20] 

Organizational  Structure 

One  of  most  common  difficulties  cited  by  WIDE  participants  in  their  surveys  has  been  the  lack  of  time  for  full 
participation  in  our  online  professional  development  courses.  It  is  difficult  to  keep  up  with  ongoing  self-reflection 
and  group  discussions,  while  fulfilling  their  full-time  job  requirements  in  school.  Although  the  asynchronous 
features  of  online  courses  provide  certain  advantages,  it  is  still  difficult  to  find  time  and  manage  other  competing 
priorities. 

It  is  interesting  to  read  the  online-survey  comments  from  teachers  regarding  this  issue.  “Our  biggest  constraint  was 
the  timing  of  each  week's  assignment  was  not  in  line  with  when  we  could  meet  as  a [WIDE]  team.  So  we  were 
always  a little  behind.  I also  personally  felt  that  I didn't  have  time  to  give  enough  feedback  to  other  participants.  It 
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was  a challenge  to  just  get  our  assignments  done.  Also  this  year  NYC  has  us  doing  an  inquiry  [team]  project  and 
that  distracted  us.”  [End-of-course  online  survey,  Fall07,  ECE-DW,  10482] 

Another  teacher  explained,  'Time'  was  always  a factor  for  this  team.  We  are  coaches  and  data  specialist,  and 
have  a very  demanding  schedule  during  the  school  day.  We  are  therefore  limited  in  our  time  to  meet, 
collaborate  and  execute  plans  with  the  students  who  are  chosen  from  different  classes. 

Times  available  for  our  team  to  meet  and  collaborate  were  not  always  in  sync  with  assignment  due  dates 
(which  we  thought  were  a bit  unrealistic).  We  tried  our  best  to  meet  deadlines,  however  with  breaks  in  the 
school  semester  and  other  work-related  commitments,  this  was  not  always  possible.  [End-of-course  online 
survey,  Fall08,  CNE-MI,  1 1582] 

As  mentioned  earlier  in  this  report,  each  school  allocated  a special  time  and  space  for  their  Harvard/WIDE  team 
course  work.  Clearly  this  set-aside  time  was  important  to  the  participants.  The  CNE  teachers,  for  example,  expressed 
their  appreciation  for  their  Thursday  after-school  time  , set  aside  to  work  exclusively  on  WIDE  course  activities. 

“We  were  given  time  only  by  our  institution  [school].  This  was  very  helpful  due  to  the  intense  workload  of  the 
course.”  [End-of-course  online  survey,  Fall08,  CNE-MI,  11585] 

Another  teacher  explained,  “The  school  Principal  was  very  helpful  and  supportive  of  our  group  in  working  on 
our  assignment.  She  allows  us  to  use  the  resources  that  we  need  for  the  course.  [End-of-course  online  survey, 

Fall08,  CNE-MI,  15113] 

Principal  Jones  noted  that  “We  know  the  routine.  We  know  it  is  on  Thursdays,  and  we  kind  of  look  forward  [to  it]. 
So,  I changed  things  [school  schedule]  around.  So  we  [the  Harvard  Team]  would  be  freed  up  on  that  Thursday.” 
Principal  Jones  also  coordinated  the  entire  PD  schedule  so  that  “once  a month,  the  3-5th  grade  teachers  meet 
together.  At  another  time,  the  whole  school  meets  together,  and  then  I give  them  time  once  a month  that  we  have  an 
hour,  almost  two  hours  of  professional  development  built  into  their  schedules,”  where  they  often  work  with  the 
student  assessment  data  or  other  WIDE-inspired  practices  including  DI. 

For  a staff  member  who  develops  professional  development  activities  at  CNE,  for  example,  the  DI  theory  helped 
differentiate  his  staff  training.  “When  I did  [staff]  training  this  year,  I made  sure  that  my  training  was  different  for 
my  staff,  because  they  each  had  different  levels  of  experience.  One  of  the  things  we  talked  about  was  child 
development  and  1 did  a whole  piece  on  different  kids  are  in  different  places,  what  you  do  about,  how  you  handle  the 
class  of  twenty  children  when  three  of  them  are  so  far  behind.  . . . So,  from  the  big  picture  standpoint.  I’m  using  the 
[DI]  concepts...  so  it’s  a little  bit  different.”  [CNE  FG2_TSNT1] 

Both  ECE  and  TAM  also  applied  the  concept  of  DI  in  their  professional  development  activities.  One  staff  developer 
at  ECE  said  that  “I  have  been  differentiating  PD  more.  We  gave  menus  such  as  the  beginner’s  or  advanced  training 
on  Smart  Board,  and  did  certain  seminars  that  were  basically  self-selecting  and  based  on  differentiation."  [ECE_FG] 
Principal  Ellis  at  TAM  also  pointed  out  that  “we  also  learned  [from  the  DI  course]  that  we  need  to  differentiate  in 
professional  development. . . For  example,  we  have  a group  of  teachers  that  they  only  just  become  new  teachers  and 
they  have  different  professional  development  from  the  rest  of  the  other  teachers,  so  new  teachers  have  four  periods 
of  PD  instead  of  three  periods  [for  other  teachers].  One  of  the  [four]  periods  is  just  to  teach  them  how  to  work  with 
the  class,  how  to  set  up  the  classroom,  those  basic  things  that  the  older  teachers  already  know.”  [TAM  Lopez  1 
Feb09] 

According  to  Principal  Ellis,  their  professional  development  time  is  “embedded  in  the  teacher’s  schedule,  so  ...  the 
number  of  instruction  periods  is  being  reduced  so  that  they’ll  be  able  to  have  those  periods  mandated  for 
professional  development.”  [TAM  Ellis  1 Feb09] 

Visionary  and  Distributed  Leadership 

Another  key  element  of  successful  systemic  educational  improvement  is  to  develop  visionary  and  distributed 
leadership  that  will  create  and  sustain  common  educational  goals  within  a system.  This  means  that  all  leadership 
competences  should  not  be  vested  in  one  leader.  Rather,  with  systemic  leadership,  all  staff  can  identify  the  critical 
vision  or  mission  of  the  system,  can  recognize  necessary  steps  and  resources  required  to  meet  the  goals,  and 
authority  and  responsibility  is  distributed  across  different  levels  of  the  system. 
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By  being  closely  aligned  with  the  school’s  key  priority  (improving  the  data  collection  and  presentation  for  the  2007- 
OS  Quality  Review),  one  CNE  teacher  called  the  DW  course  “really  really  eye  opening”  because  it  not  only  used  the 
school’s  own  assessment  data  but  offered  ongoing  opportunities  for  the  school  administrators  and  teachers  to  work 
collaboratively  to  “see  how  we  can  use  what  we’ve  gathered  and  use  it  best  to  help  the  children.”  [CNE  FG1_MI] 
Another  teacher  provided  a concrete  example  of  how  the  whole  school  started  to  change:  “you  do  a little  more  [with 
the  data],  and  then  slowly,  certain  teachers  who  were  very  resistant  and  very  vocal  about  not  wanting  to  collect  all 
this  data  and  how  it  was  a waste  of  time,  were  suddenly  saying,  “Oh!  This  kid,  he  really  changed,  and  now  I know  I 
have  to  do  this  with  him.”  To  her,  it  was  that  turn  when  a majority  of  teachers  in  the  school  “really  started  looking  at 
[the  data]  and  said  ‘ok,  what  does  that  mean  in  the  classroom,  what  exactly  do  we  do  with  all  this  information?’ 
“That  was  when  the  buy-in  started  happening,  because  people  were  able  to  see  the  reason,  they  could  see  the 
results.” 

Thanks  to  this  school-wide  systemic  effort  in  improving  the  use  of  student  data,  CNE  has  accomplished  its  goal  to 
move  the  school  from  “In  Need  of  Improvement”  in  2006-07  to  “Proficient”  status  in  the  2007-08  NYC  Quality 
Review,  as  shown  in  the  table  below. 


Review  Category 

2006-2007 

2007  - 2008 17 

School  Environment 

0.812 

A (11.2) 

Student  Performance 

0.441 

B (15.2) 

Student  Progress 

0.546 

A (42.6) 

Additional  credit 

2.25 

0.75 

Overall  Score 

In  Need  of  Improvement  - Year  1 

A (69.8)  Proficient 

[Table:  CNE  NYC  Quality  Review  point/grade] 


One  of  the  key  contributing  factors  in  this  remarkable  improvement  was  the  visionary  and  consistent  support  from 
the  school  leadership  group,  particularly  from  Principal  Jones. 

When  asked  to  describe  her  leadership  style,  Principal  Jones  explained  that,  “It  is  important  to  be  [a  leader]  but  it  is 
also  more  important  to  be  a part  of  the  group.  Instead  of  saying  I am  appearing  as  a principal.  I am  different  from  all 
of  you.  ..  For  me,  it  is  important  to  take  ownership  [as  a leader]  and  being  part  of  that  team  down  here.  And  knowing 
and  seeing  things  at  their  level.”  “There  has  to  be  a way  that  we  mutually  can  share  things  because  if  I could  not 
share,  then  I am  not  a good  leader,”  added  Principal  Jones.  Principal  Jones  sees  herself  as  someone  who  “helps 
people  and  asks  for  their  input.”  “It  is  not  about  me  directing  them.  I act  more  as  a facilitator,”  added  Principal 
Jones.  In  fact,  one  teacher  noted  that  “I  think,  like  one  of  the  teachers  said  to  me,  because  you  don’t  usually  see  Ms 
Jones  in  another  light  other  than  as  principal.  In  that  [WIDE]  course,  she  is  like  one  of  the  guys.  So  it  was  very  good 
for  the  teachers  to  see  her  and  to  communicate  with  her  on  a level  that  they  would  not  normally  communicate  with 
her.” 

Principal  Jones’s  creative,  energetic,  and  committed  leadership  was  fully  recognized  by  being  one  of  the  five 
recipients  of  the  2007  Sloan  Public  Service  Awards,  selected  among  more  than  250,000  public  service  employees  in 
the  NYC  government.  According  to  a CNE  teacher’s  description  of  Principal  Jones,  appeared  in  the  published 
booklet  for  this  award,  she  is  “a  tireless  advocate  for  children,  she  also  pushes  teachers  to  excel  in  their  professional 
and  personal  development;  she  gives  us  leeway  to  do  our  jobs  creatively  and  creates  a sense  of  unity,  commitment 
and  ownership  in  the  programs  she  promotes.” 

Principal  Miller  vividly  recalled  his  first  arrival  at  ECE,  which  was  a failing  school.  “When  I got  here,  there  was  a 
lot  of  finger  pointing:  the  teachers  blame  the  parents,  the  parents  blame  the  teachers.  It  was  an  ongoing  cycle  and  it 
was  easier  to  blame  than  to  do.  I set  very  high  expectations.  What  was  really  frightening  was  that  everyone  told  me 


Please  note  that  the  NYC  Quality  Review  implemented  a grade  system  (A  to  F)  in  the  school  year  of  2007-2008. 
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that  it  is  ‘such  a great  school.’  That  really  scared  me.  During  lunch,  I went  to  a copy  store  and  blew  up  our  test 
scores  into  a six  feet  poster.  I hung  them  all  over  the  school  and  asked  the  teachers:  ‘would  you  send  your  child  to 
this  school?”  Awakening. . . [I  answered  that]  ‘I  would  not  send  my  child  to  this  school.’  I am  not  going  to  be 
satisfied  until  each  one  of  your  child,  nephew,  and  best  friend’s  child  would  want  to  come  to  this  school.’  That  is 
basically  how  we  get  started  [changing].” 

Principal  Miller  described  that  “it  really  set  the  expectation  high  and  let  everyone  know  that  this  place  is  about 
moving  children  and  working  with  children.  . . . This  was  not  a factory  to  punch  in  and  punch  out.  I needed  to  make 
that  change.”  One  of  his  teachers  explained  that,  at  the  beginning,  “teachers  would  ask  questions  about  parking 
permits,  leaving  early  to  go  to  a doctor”  to  Principal  Miller.  He  intentionally  ignored  these  requests  and  instead 
Assistant  Principal  talked  to  the  teachers  that  “If  you  ever  want  to  talk  to  Principal  Miller,  it  needs  to  be  around 
student  outcomes  and  achievement.”  Principal  Miller  pointed  out  that  “it  was  a defining  moment,  a statement.” 

As  of  the  2004-2005  Report  Card,  ECE’s  designation  was  changed  from  “Schools  In  Need  of  Improvement”  to 
“School  in  Good  Standing.”  Since  then,  as  shown  in  the  graph  below,  ECE  has  continued  striving  with  significant 
academic  growth,  with  percentages  signifying  the  percent  of  students  are  or  above  standard  (Levels  3 & 4)  on  NY 
State  standardized  exams.  “This  is  where  anyone  that  looks  at  the  data  will  tell  you  with  the  wrong  zip  code,  with 
the  wrong  color,  with  the  wrong  socio-economic  status  for  children  to  be  successful,  and  we  break  that  mold  here. . 
This  is  a school  where  all  children  are  successful,”  proudly  said  Principal  Miller. 


Review  Category 

2006-2007 

2007  - 2008 

School  Environment 

13 

A (10.4) 

Student  Performance 

25.9 

A (23.1) 

Student  Progress 

32.1 

B (30.2) 

Additional  credit 

1.5 

0.8 

Overall  Score 

A (72.4) 

In  Good  Standing 

A (64.5) 
Well-developed 

[Table:  ECE  NYC  Quality  Review  point/grade] 


When  asked  to  describe  his  leadership  style.  Principal  Miller  said  “Different!  Very  much  out  of  the  box!  I don’t 
consider  myself  as  a leader.  I spent  most  of  my  time  in  classrooms.  The  work  of  the  teachers  in  classroom  is  my 
work.  I see  myself  in  the  classroom  and  my  philosophy  has  changed  drastically  in  the  last  six  years.”  [ECE_Miller] 

As  ECE’s  2008-09  School  Comprehensive  Educational  Plan  (CEP)  indicates,  the  school  has  gone  through  a 
dramatic  improvement  “through  determination,  newly  developed  collaborations,  and  a strong  commitment  from 
staff  and  parents.”  According  to  this  plan,  ECE  is  most  proud  of  these  two  accomplishments:  1)  “increased  student 
achievement  brought  about  by  the  improvement  of  the  quality  of  assessment  and  instruction”  and  2)  “the  creation  of 
a truly  collaborative  teaching  environment,  as  well  as  a comprehensive  inclusive  student  environment.” 

The  previous  section  described  Principal  Ellis  and  her  leadership  team’s  changes.  After  going  through  various  steps 
to  improve  engagement  and  communication.  Principal  Ellis  described  the  current  meeting  environment  in  this  way: 
“Before  the  WIDE  classes,  it  was  unbelievable.  At  first  we  [the  leadership  team]  were  dictating.  The  leadership  team 
would  tell  everything. . .this  is  what  you  [a  teacher]  do.  No  input.  It  just  had  to  be  done.  ...  [if  it  doesn’t  happen]  I 
used  to  think,  why  do  the  teachers  don’t  do  this?  Why  do  they  not  do  it  even  after  we  explained  it?  So  we  used  to  see 
it,  “Okay,  they  don’t  want  to  do  it.”  But  after  being  in  those  classes — Leading  for  Understanding — we  began  to 
think,  maybe  they  don’t  know  how  to  do  it,  not  that  they  don’t  want  to  do  it.  ...  Let  them  participate  in  the  input  in 
how  to  make  changes.  Part  of  it,  they  will  get  engaged. . . Now  it’s  different.  . . . For  example,  for  the  6th  grade,  we 
have  two  teachers  who  are  in  charge  of  the  whole  6th  grade.  They  plan  the  PD  and  everything.  We  [the  leadership 
team]  just  go  and  sit  down  and  listen.”  [TAM  Ellis  1 Feb09]  “It  definitely  changed  me  as  an  administrator  and  now 
it’s  changing  my  staff,”  continued  Principal  Ellis. 
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The  following  table  shows  how  TAM  improved  its  Quality  Review  results  for  the  last  two  years  by  moving  its  status 
from  “Planning  for  restructuring”  to  “Well-developed.”  Its  2007-08  Progress  Report,  reported  that  “the  school  met 
100%  of  its  improvement  target  from  last  year.” 


Review  Category 

2006-2007 

2007  - 2008 

School  Environment 

6/15 

D (4.1/15) 

Student  Performance 

9.8/30 

B (14.0/25) 

Student  Progress 

17.1/55 

C (25.7/60) 

Additional  credit 

0.8 

0.8/15 

Overall  Score 

D (33.7) 

Planning  for  Restructuring 

C (44.6/100) 
Well-developed 

[Table:  TAM  NYC  Quality  Review  point/grade] 


VII.  Conclusion 

“If  it  [WIDE  World]  works  here,  it  works  everywhere !”  claimed  Principal  Miller.18 

The  three  NYC  schools  in  this  study  were  not  the  best  schools  few  years  ago.  Thanks  to  their  visionary,  coherent, 
and  persistent  efforts  to  improve  student  academic  achievement,  each  school  made  significant  progress  in  classroom 
teaching,  student  learning,  and  overall  school  performance.  This  research  report  used  the  WIDE’s  Systemic 
Educational  Improvement  Model  to  capture  and  analyze  the  ways  in  which  each  school  faced  its  own  challenges  and 
accomplished  important  educational  improvement  over  the  last  two  years. 

A majority  of  the  NYC  participants  in  this  study  reported  that  their  WIDE  experience  helped  them  grow,  not  only  as 
individual  teachers  or  leaders,  but  as  collaborative  members  of  a whole  school.  As  this  paper  illustrates,  systemic 
educational  improvement  requires  a thoughtful  coordination  of  critical  teaching  and  learning  activities  across  all 
levels  of  personnel  in  a school. 

In  conclusion,  we  would  like  to  recapitulate  the  five  strongest  areas  of  WIDE’s  impact  on  the  three  NYC  schools. 

1.  Improved  understanding  and  enactment  of  high-quality,  effective  teaching,  learning,  and  leadership 
Based  on  long-term  participation  in  WIDE  courses,  the  members  of  these  schools  have  acquired  improved 
understanding  and  enacted  high-quality  and  effective  teaching,  learning,  and  leadership.  Throughout  the  study,  they 
demonstrated  their  capability  to  use  common  vocabulary  (such  as  the  TfU  framework)  and  articulate  shared  norms 
for  high-quality  instructions  and  outcomes. 

2.  Improved  engagement  and  collaboration  among  the  school  members 

We  collected  abundant  evidence  of  improved  levels  of  engagement  and  collaboration  among  the  school  members, 
including  examples  related  to  self-esteem,  professionalism,  and  collegiality.  These  changes  indicate  that  the  culture 
at  the  NYC  schools  has  shifted  from  an  autocratic  to  a more  democratic  learning  and  working  environment. 

3.  Increased  capacity  of  Inquiry  process  across  levels 

By  establishing  a professional  learning  community  through  the  WIDE  coursework,  these  teachers  and  leaders  are 
now  accustomed  to  ask  more  questions  of  one  another  and  their  students.  By  regularly  reflecting  on  their  knowledge 
and  practices,  these  educators  regard  themselves  as  life-long  learners  and  are  eager  to  prepare  their  students  to  do  the 
same. 

4.  Visionary  and  distributed  leadership 

The  leaders  in  this  study  are  extraordinary  individuals.  They  have  demonstrated  their  capacity  to  lead  their  schools 
with  a clear  vision,  collective  decision-making  mechanisms,  shared  responsibilities,  and  untiring  support  to  make  the 
WIDE  experience  as  successful  as  possible.  These  leaders  also  increased  the  leadership  capacity  in  their  schools  by 
collaborating  with  teachers  and,  in  the  process,  creating  new  leaders. 


From  a personal  communication  with  Dr.  Joo  during  her  research  visit  in  February  2009. 
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5.  Data-driven  educational  improvement 

In  order  to  address  the  NYC’s  accountability  requirements,  these  schools  made  tremendous  progress  using  their 
student  data  for  school-wide  educational  improvement.  They  worked  strategically  to  understand,  collect,  analyze, 
and  present  their  assessment  data  for  the  annual  Quality  Reviews. 

At  WIDE  World,  we  are  proud  of  our  successful  partnership  with  the  NYC  schools  and  grateful  to  the  Harvard  Club 
of  New  York  Foundation  for  supporting  the  last  two  years  of  educational  improvement. 

“We  are  talking  about  the  [WIDE]  graduation,  but  it  is  a celebration!”  Principal  Ellis  said  excitedly.  [TAM  Ellis] 
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Exemplary  Use  of  Technology  in  k-12  Education  in  Saudi  Arabia: 

Dar  Al-Fikr  Private  School 

Abdulrahman  Kamal 
Kansas  State  University 
2215  Prairie  Glen  PI 
Manhattan,  KS  66502 
USA 

Descriptors:  Educational  Technology,  Saudi  Arabia 

Introduction 

Educational  technologies  are  rapidly  changing  teaching  and  learning  environments  in  developing  countries 
(US  Fed  News  Service,  2008).  Saudi  Arabia  is  an  example  of  a developing  country  that  has  accelerated  the 
implementation  of  educational  technologies  ( Al- Abdulkareem,  2008;  Al-Yousif,  2008).  While  Saudi  Arabia 
pledged  26  percent  of  its  budget  since  2006  for  building  2,600  K-12  schools  ( Al-Faisal,  2006),  little  has  been  written 
about  K-12  education  in  Saudi  Arabia  outside  of  the  Kingdom. 

Research  about  K-12  education  in  Saudi  Arabia  has  been  limited.  Only  two  dissertations  were  written  on 
K-12  education  years  ago  ( Al-Hareky,  1983;  Issa-Fullata,  1982).  As  a result,  the  present  state  of  K-12  education  in 
Saudi  Arabia  is  largely  unknown  in  the  United  States  and  the  West,  in  general,  since  the  Kingdom  does  not  provide 
much  information  on  the  subject  (Prokop,  2003).  This  presentation  will  discuss  some  of  the  barriers  to  improving  the 
use  of  technology  in  K-12  education  and  show  the  reality  of  implementing  educational  technologies  in  Saudi  Arabia 
through  some  initiatives  that  have  been  established  by  the  Ministry  of  Education.  It  also  demonstrates  an  exemplary 
use  of  educational  technology  at  the  Dar  Al-Fikr  private  school  in  Jeddah.  It  is  hoped  that  this  presentation  will 
provide  a basis  for  understanding  the  many  changes  that  are  taking  place  in  Education  in  Saudi  Arabia,  as  a result. 

Curriculum  Development 

In  2002,  the  Ministry  of  Education  has  established  “The  General  Project  for  Curriculum  Development”  to 
reconstruct  and  improve  curricula  for  different  subjects.  One  of  the  project’s  aims  emphasizes  the  importance  of 
implementing  technology  in  this  reconstruction  process. 

Providing  effective  methods  accomplishes  educational  policy  with  integration.  This  is  done  by 
effectively  interacting  with  new  educational  technologies,  benefiting  from  experiences  of  others, 
specifying  required  skills  to  be  learnt  by  students  at  every  educational  level,  linking  information 
with  general  life,  developing  critical  thinking  methods,  and  developing  required  skills  and 
essentials  for  productive  work  (curriculum  development  in  S.A,  2006,  Para.  3). 

The  Ministry  of  Education's  ten-year  strategic  plan  was  established  in  2004.  It  stated  that  its  Information  and 
communication  technology  (ICT)  goal  is  “to  develop  the  infrastructure  of  information  and  communication 
technology  and  its  employment  in  education  and  learning”  (Computer  and  Information  Center,  2008,  Para.  1). 

Dissertations  on  Saudi  Arabian  K-12  Education 

A recent  search  of  the  ProQuest  database  of  Dissertations  and  Theses  on  February  16th,  2009,  found  only 
few  studies  conducted  on  higher  education  e-learning  in  Saudi  Arabia.  Two  studies  were  conducted  on 
implementing  educational  technology  in  k-12  education.  Al-Hareky(  1983)  studied  the  effectiveness  of  modern 
educational  technology  on  the  mathematics  performance  of  elementary  students  in  Saudi  Arabia  (instructional  TV, 
computer  based  instruction).  The  study  found  that  there  was  significant  difference  in  the  total  mean  of  achievement 
scores  was  higher  for  the  computer  group  than  the  control  group.  Also,  the  total  mean  of  achievement  scores  of 
television  group  was  significantly  better  than  the  control  group.  The  study  also  concluded  that  there  was  a positive 
attitude  among  the  majority  of  students  and  teachers  toward  computer-based  instruction  and  instructional  television. 
The  study  also  found  no  significant  difference  between  the  total  mean  achievement  scores  for  males  and  females  in 
each  group. 
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Issa-Fullata  (1982)  studied  the  impact  of  modernizing  instruction  through  educational  technology  on 
students’  achievement  in  Saudi  Arabia.  Four  instructional  treatments  were  used:  a control  group;  a regular  teaching 
method  with  a classroom  teacher,  an  experimental  group  1 with  an  assisted  tape/slide  instructional  program  with  the 
classroom  teacher,  and  experimental  group  2 with  a self-contained  instructional  tape/slide  program  without  the 
classroom  teacher  participation.  Two  instruments  were  used  to  measure  students’  achievement:  a paper-pencil  and  a 
visual/tape  format.  Analysis  of  Variance  showed  that  only  experimental  group  1,  the  paper-pencil  subgroup  showed  a 
significant  difference  at  P < .05.  Teachers  also  showed  positive  attitudes  toward  educational  technology  and 
instructional  design. 


Barriers  to  Using  Technology  in  Saudi  Arabia 

There  are  several  barriers  that  need  to  be  overcome  to  reach  the  desired  implementation  and  spread  the 
benefits  of  implementing  educational  technologies  to  touch  all  the  Saudi  students.  These  barriers  include: 

Weakness  of  infrastructure  especially  communication  infrastructure. 

Need  for  technology  specialists. 

Lack  of  technological  knowledge  and  skills  among  teachers  and  administrators. 

English  language  barrier  (for  example,  most  web  2.0  tools  are  in  English). 

High  coast  of  technology.  ( Al- Abdulkareem,  2008) 

With  this  lack  of  information  about  the  current  reality  of  implementing  educational  technologies  in  Saudi 
Arabia  educational  system,  as  well  as  these  barriers,  more  studies  are  needed  to  investigate  the  reality  of 
implementing  educational  technologies  and  to  help  in  overcoming  these  barriers. 

New  Educational  Programs  in  Saudi  Arabia 

In  order  to  fulfill  the  goal  of  implementing  technologies  in  K-12  education,  the  Ministry  of  Education  has 
started  different  projects.  For  example: 

Obikan  & Intel  Project: 

Obikan  Research  and  Development  Company  (ORD),  in  cooperation  with  Intel,  has  developed  an  e-Learning 
portal  using  Intel's  School  Learning  Software,  called  'skooof  to  support  school  curriculum  in  Saudi  Arabia.  Skoool 
concentrates  on  secondary  Math  and  Science  curricula.  “It  includes  a wide  range  of  interactive  educational 
resources  that  aim  to  complement  students'  schoolwork  and  home  study  by  showing  them  how  to  save  time  and 
improve  their  academic  performance”  (Saudi  company  Obeikan  to  develop  e-learning  portal  utilizing  Intel’s  school 
learning  software,  2005.  Para.  2).  Abdullah  Obikan,  an  engineer  at  ORD,  said  that  “by  adapting  our  interactive 
learning  and  supplementary  material  to  the  Arabic  language,  we  hope  to  help  increase  technology  literacy  and 
academic  achievement  in  the  Kingdom”  cited  in  ((Saudi  company  Obeikan  to  develop  e-learning  portal  utilizing 
Intel’s  school  learning  software,  2005,  Para.  4). 

King  Abdullah  Project  for  Educational  Development  (KAPED): 

The  largest  and  the  most  recent  program  is  “King  Abdullah  Project  for  Educational  Development”,  which 
has  a budget  of  about  $ 2.4  billion  and  aimed  to  guarantee  the  availability  of  a highly  skilled  and  motivated  work 
force  in  the  future.  The  project  will  “begin  with  creating  a high-tech  classroom  environment  in  the  Kingdom  in  six 
years.  More  than  400,000  teachers  will  be  trained  to  handle  classes  in  the  high-tech  style”  (Ministry  of  Education  in 
S.A,  2008). 


Figure  1:  (KAPED)  logo,  (Tatweer,  2008) 

The  project  has  started  since  the  last  academic  year  by  assigning  fifty  high  schools  among  the  country  (25 
male  and  25  female  schools)  as  a pilot  schools  to  implement  the  project.  These  schools  have  been  equipped  by  the 
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required  advanced  technologies  to  facilitate  what  is  called  the  smart  classrooms.  These  facilities  include  smart  white 
board,  one-one  laptop,  data  projector,  digital  video  camera,  and  high  speed  internet  (Tatweer,  2008).  Each  school  is 
also  equipped  with  access  to  virtual  laboratories  and  museums,  digital  library,  and  the  required  facilities  for  the  extra 
curricula  activities  (Tatweer,  2008).  Also,  the  project  adopts  the  project  based  learning  (PBL)  as  a learning  strategy 
to  support  the  student-centered  education  (Tatweer,  2008).  In  addition,  these  changes  in  the  learning  strategies 
require  change  in  the  assessment  tools  to  be  used  as  “assessment  for  learning”  instead  of  “assessment  of  learning”. 
Therefore,  the  project  adopts  the  authentic  assessment  tools  such  as  performance  assessment  instead  of  the 
standardized  tests  (tatweer,  2008).  Finally,  the  professional  training  for  teachers  and  school  principals  (male  and 
female)  to  prepare  them  for  the  skills  that  are  required  to  implement  the  project  properly  plays  important  role  in  the 
aimed  success  for  this  project.  Last  year  during  the  summer  vacation  1700  teachers  and  principals  were  trained  for 
the  use  of  the  new  technology  and  the  new  learning  and  assessments  strategies  (Ministry  of  Education  in  S.A,  2008). 


Figure  2:  One  to  one  laptop  program  (Tatweer,  2008) 


Figure  3:  PBL  (Tatweer,  2008) 


Dar  Al-Fikr  K-12  Private  School 

The  K-12  Dar  Al-Fikr  Private  School  is  considered  to  be  an  exemplary  model  of  implementing  educational 
technologies  K-12  in  Saudi  Arabia.  The  school  curriculum  is  based  on  IT-assisted  interactive  learning.  All  subject 
materials  are  computerized  and  loaded  onto  the  school  network.  The  school  teaches  additional  subjects  that  are  not 
taught  at  Saudi  public  schools,  such  as  Life  Skills,  Design  & Technology,  Thinking  Skills,  and  Community  Service. 
One  unusual  aspect  of  Dar  Al-Fikr’ s curriculum  is  that  the  teaching  is  done  in  both  Arabic  and  English,  which 
makes  students  benefit  from  technological  resources  and  tools  more  easily,  since  many  web  tools  are  in  English. 
Moreover,  the  school  uses  a full  electronic  administrative  system.  This  is  unusual  for  public  schools  in  Saudi  Arabia. 
(Dar  Al-Fikr  private  school,  2008) 
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Figure  6:  Art  class  at  Dar  Al-Fiker  Girl  School 


Figure  7:  Quran  class 
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Abstract 

This  study  conducted  to  investigate  novice  instructional  designers’  learning  experiences  in  a game  design  course 
where  designers  were  involved  in  designing  and  developing  educational  games  and  to  investigate  the  challenges  and 
interaction  that  were  taken  place  in  a game  development  process.  This  study  utilized  Engestrom’s  (1999)  activity 
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theory  as  its  main  theoretical  framework  to  better  understand  the  interaction  within  the  components  of  activity  and 
revealing  challenges  and  experiences.  All  components  of  activity  theory  were  examined  to  how  these  components 
influence  each  other.  From  those  interactions  what  outcomes  are  obtained  by  novice  instructional  designers  and 
what  challenges  they  have  was  obtained  from  this  examination.  Activity  system  approach  also  provide  an  evaluation 
method  for  this  complex  environment. 


Introduction 

Instructional  designer’s  skill-set  and  knowledge  that  needs  to  be  applied  in  educational  contexts  have  changed  and 
varied  over  time  (Davidson-Shivers  & Rasmussen,  2002).  Project  based  and  context  dependent  applications  are 
some  exemplary  methods  that  have  been  used  to  educate  instructional  designers  in  the  academia  (Davidson-Shivers 
& Rasmussen,  2002).  Considering  wide  range  of  methods  in  educating  instructional  designers,,  evaluation  criteria 
need  to  be  developed  to  improve  the  curriculum  of  academic  programs,  the  content  of  the  courses  and  self- 
assessment  of  trainers  (Bratton,  1990). Teaching  ID  in  the  context  of  educational  game  design  has  been  gaining  more 
interest  and  some  models  have  been  developed  for  teaching  instructional  design  by  means  of  designing  instructional 
games  (Akilli  & Cagiltay,  2006).  Game  design  for  learning  approach  have  been  used  in  education,  and  this  is 
supposed  to  more  educative  because  it  requires  inquiry,  research,  analysis  and  synthesis  which  are  desired  issues  in 
education. 

The  main  purpose  of  this  study  was  twofold:  first,  to  investigate  novice  instructional  designers’  learning  experiences 
in  a game  design  course  where  designers  were  involved  in  designing  and  developing  educational  games  , secondly 
to  investigate  the  challenges  and  interaction  that  were  taken  place  in  a game  development  process. 

Theoretical  framework 

This  study  utilized  Engestrom’ s (1999)  activity  theory  as  its  main  theoretical  framework  to  better  understand  the 
interaction  within  the  components  of  activity  and  revealing  outcomes.  Such  inquiry  requires  deep  understanding  of 
the  context  itself  and  the  participation  into  that  context  (Engestrom,  1999;  Eskola,  1999).  This  theory  was  selected 
due  to  its  capability  for  analyzing  “object-oriented,  collective,  culturally-mediated  human  activity  or  activity 
system”  (Engestrom,  1999,  p.  9). 

Activity  theory  includes  subject,  object,  tools,  rules,  community  and  division  of  labor.  In  the  context  of  this  study, 
the  components  of  activity  theory  were  formed  as  shown  on  Figure  1 . All  of  the  components  were  dynamic  and 
students  were  assigned  several  roles  during  the  project. 

Context  and  Participants 

In  this  instructional  game  design  case  study  23  undergraduate  students,  facilitators,  target  group  and  people  who  are 
facilitated  by  students  to  reach  resources  related  game  design  engaged  in  development  of  a game-like  learning 
environment.  To  reveal  the  interaction  between  all  components  of  the  activity  system,  all  the  externalized  tools  such 
as  texts,  data  and  discourse  of  all  interactions,  observations  and  products  in  collaborative  social  environment  were 
observed  and  analyzed  using  activity  theory  framework. 
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Tools 


Course  resources,  game 
development  tools,  all  facilities 
related  course 


course,  group  contracts,  Course  instructor  and  assistants,  target  students,  all  Developers,  all  roles  that 

tool  limitations  people  with  whom  students  facilitated  for  their  students  take  during  the 

projects  pioject 


Figure  1 : Components  of  activity  theory  in  the  case  study 

The  course  was  designed  using  the  FID2GE  ID  model  ( Akilli  & Cagiltay,  2006  ) which  was  developed  for  game- 
like learning  environments.  FID2GE  model  promises  a fuzzy  logic  dynamic  design  processes,  designs  are 
participatory  design  products,  continuous  formative  evaluations,  promotion  in  communication  processes  and  time 
planning.  Students  in  this  course  were  engaged  to  development  of  a virtual  university  campus.  All  groups  developed 
a game  environment  related  to  one  part  of  a university  campus  (library,  medical  center,  technopolis,  etc)  and  each 
group  prepared  an  instruction  to  help  target  learners  who  were  newcomer  students  understand  how  to  get  benefit 
from  the  specific  part  of  the  campus  teach  that  how  target  learners  can  facilitate  with  that  part  of  the  campus. 

Sample  of  the  main  study  was  23  undergraduate  students  who  were  in  their  senior  year  and  enrolled  in  an 
undergraduate  game  development  course.  Researchers  selected  three  groups  (11  students)  to  make  deep  observations 
and  interviews.  Groups  were  selected  based  on  instructors’  insights  from  previous  courses  and  students' 
performances  during  the  projects.  Groups  used  Active  Worlds  Virtual  Environment  (AWVE)  as  a development 
platform  for  their  game-like  learning  environments.  In  this  environment  each  group  had  a limited  space  to  put  their 
3D  environment.  AWVE  includes  three  parts.  In  3D  interface  students  put  3D  objects  and  they  could  make  them 
interactive.  Also  there  is  a web  part  that  students  put  stories  of  characters  in  the  game  or  story  of  environment. 

Lastly  AWVE  includes  chat  tool  that  provides  communication  with  AWVE  world  community.  Thus  facilitators 
could  trace  groups  work  by  means  of  this  virtual  world. 
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Data  Collection 


During  students'  weekly  meetings,  semi  - structured  interviews  were  conducted  with  the  groups  to  explore  their 
instructional  design  processes  and  the  problems  they  encountered.  At  the  end  of  the  semester,  six  voluntary  students 
were  interviewed  individually  about  their  learning  experiences  in  the  course  focusing  on  the  challenges,  strengths 
and  weakenesses,  and  lessons  that  students  learned  in  the  course.  Researchers  also  attended  all  classes  for 
observation.  The  study  took  15  weeks  to  be  completed.  Within  these  15  weeks  of  study,  students  attended  two-hour 
classroom  sessions  and  participated  in  weekly  meetings  with  researcher  and  other  facilitators.  Data  analysis  was 
divided  into  two  parts:  the  first  part  focused  on  the  challenges  and  interaction  between  all  components  of  the  activity 
system  while  the  second  part  focused  on  the  outcomes  and  experiences  analyzed  in  the  perspectives  of  the  subject  of 
the  activity  system. 


Results 


Challenges  in  Activity  System 

Tool  limitations  affected  novices’  motivation  in  design 

In  this  activity  system  there  are  lots  of  tools  like  previous  experience  on  group  working,  projects  and  games,  lab 
hours,  theoretical  part  of  the  course,  reports,  report  templates,  AWVE  as  development  tool,  resources  to  develop 
content  and  communication  tools,  etc.  Results  showed  that  especially  AWVe  environment  played  a big  role  in 
quality  of  projects  and  challenges.  The  limitation  of  the  AWVE  features  decreased  designer  students’  motivation  to 
build  interactive  3D  game.  Students  had  to  create  a 2D  web  based  environment  in  the  virtual  network  to  make  their 
target  group  play  in  3D  part.  At  the  beginning  of  the  course  students  were  very  motivated  to  develop  an  enjoyable 
and  challenging  3D  game.  But  because  target  group  had  to  be  dependent  to  web  based  part,  designers  thought  that 
they  could  not  provide  an  enjoyable  game  that  provide  a flow  (Csikszentmihalyi,  1990)  in  that  environment.  One  of 
student  put  into  words  this  problem  by  saying  “3D  part  is  nothing  without  2D  part,  player  just  walk  around  the 
virtual  world  if  he  is  not  engaged  some  operations  by  means  of  2D  part”.  Also  students  could  not  find  all  resources 
to  design  3D  part,  and  among  300  million  3D  objects  finding  necessary  one  was  very  difficult  for  students.  One  of 
the  students  suggested  that  there  should  be  a search  engine  to  find  necessary  3D  objects.  Most  of  students  lost  their 
time  to  just  find  one  piece  of  3D  object. 

Another  challenge  that  affected  students’  motivation  in  designing  3D  game  was  related  to  the  issue  of  control  of  the 
game.  We  found  that  students  could  not  use  the  AWVE  to  offer  several  challenged  choices  in  the  game  while  target 
group  can  just  follow  a sequence  of  instruction.  Those  problems  caused  reduce  of  motivation  of  students  and 
increased  communication  in  community. 

Novices’  tended  to  ignore  tool  limitations  before  development  phase 

Students  had  to  prepare  reports  before  they  start  to  project.  Some  of  students  were  not  aware  of  instructional  design 
processes  and  therefore  they  thought  these  reports  are  needless.  When  they  pay  less  attention  in  design  they  had  lots 
of  difficulties  while  developing  their  projects.  Students  were  found  to  be  influenced  by  the  desire  to  developed  the 
3D  game  that  they  did  not  realize  the  limitation  of  the  AWVE  in  the  analysis  phase  and  they  ignore  some 
instructional  and  technical  issues  in  design  phase.  Consequently,  they  had  to  modify  their  initial  game  design.  As 
one  student  said  “as  an  instructional  designer  we  planned  totally  different  things  but  because  of  the  tool  we  had  to 
develop  a thing  that  is  not  like  a game”.  This  also  showed  that  students  tend  to  ignore  instructional  design  rule  of  the 
course,  because  FID2GE  model  proposes  that  each  step  includes  some  experience  for  next  step  (Akilli  & Cagiltay, 
2006).  For  example  students  had  to  develop  some  prototypes  while  they  were  making  their  design  and  writing 
scenario. 
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Being  relied  on  community  to  solve  problems 

In  this  project,  community  consisted  of  group  members,  users  of  Active  Worlds  Virtual  Environment  (AWVE), 
target  group  (newcomers  to  the  university),  instructors,  facilitators,  experts  and  peers.  Students  interacted  with  all  of 
them  during  the  game  development  project  using  participatory  game  design  approach.  The  interaction  within  the 
community  was  supported  by  the  Active  Worlds  Virtual  Environment  (AWVE).  Students  received  support  not  only 
by  their  own  learning  community  in  the  course,  but  also  from  users  of  the  AWVE  who  shared  their  knowledge  the 
construction  of  3D  world.  Groups  seek  a solution  for  their  problems  by  means  of  instructors  and  facilitators.  If  they 
could  find  an  experienced  user  in  AWVE  environment  they  facilitated  from  them.  For  example  one  of  students 
stated  his  experience  by  saying  “Group  member  stated  that  “we  met  with  a user  who  have  . . . nickname,  he  had 
made  lots  of  things  with  a basic  account,  he  helped  us,  we  took  Roller  Coaster  idea  from  him. . ..”.  Quality  of  object 
most  of  time  changed  because  of  tool  and  motivation  of  group  members.  It  was  observed  that  if  students  work  with 
target  group  effectively  their  motivation  increase  but  sometimes  they  did  not  communicate  well  and  they  could  not 
reflected  their  findings  to  the  game. 

Facilitators  had  an  effect  to  guide  their  projects  and  they  also  checked  rules  during  the  project.  In  addition,  we  found 
that  facilitators  played  crucial  role  in  helping  to  solve  group-related  problems  such  as  risk  management  issues  and 
consequently  guaranteed  the  quality  of  object  (game-like  learning  environment).  Groups  defined  some  rules  at  the 
beginning  of  the  projects  and  they  sign  a contract  which  consisting  of  responsibilities  of  members,  timetables,  risk 
management  and  penalties.  Students  preferred  instructor  control  for  penalties.  This  can  be  interpreted  as  students 
need  a more  professional  control  mechanism  to  make  them  work  on  a project. 

Communication  with  facilitators  and  other  community  members  also  highly  involved  with  quality  of  work.  For 
example  the  groups  interacted  with  facilitators  were  more  successful  groups  and  in  these  groups  all  group  members 
showed  up  in  each  weekly  meeting.  Researchers  realized  that  some  groups  come  to  weekly  meeting  just  because  it  is 
a rule  of  the  course,  but  successful  groups  came  additional  meetings  to  ask  some  strategies  and  solve  their  problems. 
Successful  groups  also  did  their  group  meetings  rigidly  and  reported  it  to  facilitators. 

The  importance  of  rules 

Despite  of  the  course  rules,  we  found  the  need  to  develop  community  rules  to  help  novices  produced  quality  work. 
For  example  in  observations  it  was  realized  that  one  of  the  groups  developed  their  project  with  a totally  different 
instructional  approach  than  the  one  suggested  in  the  course  rules.  Not  only  this  influenced  the  quality  of  their 
project,  it  also  affected  their  knowledge  on  effective  instructional  approaches  in  game  design.  This  case  mainly 
caused  because  of  students  previous  experience  with  a similar  course.  Students  mainly  used  templates  and 
knowledge  coming  from  a multimedia  development  course  which  was  given  almost  same  structure.  On  the  other 
hand  facilitator  also  could  realize  this  problem  and  if  this  problem  was  realized  by  their  facilitator  on  time,  quality  of 
work  could  change. 

Design  of  the  course  also  influenced  division  of  labor,  which  in  this  paper  means  “roles”  because  it  requires  group 
working,  work  with  a real  target  group.  On  the  other  hand  students  preferred  different  division  of  labor;  some  groups 
preferred  to  work  on  each  labor  (i.e.  role)  equally,  and  others  preferred  to  give  each  group  members  a labor  (i.e. 
role).  Groups  that  assigned  one  labor  (i.e.  role)  for  each  group  member  encountered  more  problems  because  one 
member’s  work  affected  the  entire  group  process  However,  we  found  that  the  course  rules  contributed  to  solving 
group  problems.  For  instance,  student  designer  of  a group  who  was  not  completing  his  tasks  would  faced  certain 
actions,  thus  other  group  members’  rights  were  protected  by  rules.  But  still  students  hesitated  to  say  these  kind  of 
problems  to  facilitators  and  they  expected  facilitators  interpret  the  problem  in  group  working.  In  this  case  facilitators 
need  to  know  everything  in  group  contract  and  understand  problems  by  making  in-depth  observations  to  reveal 
problems. 
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Experiences  and  Outcomes 
Experiences  of  Students 

Students  had  to  follow  an  instructional  design  process  thus  they  conducted  several  analyses  like  learner,  needs, 
content,  context,  and  tool.  For  all  stages  of  design  they  submit  a report  to  summarize  their  work.  In  evaluation  stage 
they  conduct  usability  evaluations  and  evaluations  related  quality  of  instruction.  In  development  stage  they 
designed  a 3D  environment  and  web  sites  that  were  integrated  to  3D  part.  For  group  working  they  also  had  some 
roles  like  leader,  communication  person,  scenario  write,  and  reporter. 

Changes  in  roles 

Data  from  interviews  and  observations  revealed  that  during  the  game  design  project,  students’  roles  changed 
between  being  a coder,  designer,  developer,  task  analyst,  reporter,  narrator,  usability  test  designer,  leader, 
coordinator,  resource  manager.  In  addition,  we  found  that  students  were  developing  other  soft  skills  such 
asconducting  research,  communication,  conducting  needs  assessment,  and  analyzing  target  group  while  engaging  in 
this  project.  For  instance,  they  conducted  usability  testing  for  their  designs  which  then  affect  their  communication 
skills. 

Online  community 

Another  important  finding  was  the  role  of  Active  Worlds  as  a venue  to  build  online  community  among  students 
while  simultaneously  functioned  as  tool  to  develop  game.  Using  Active  Worlds,  students  coordinated  meeting  with 
their  target  groups  and  continuously  worked  with  group  members  in  the  process  of  selecting,  modifying  and 
developing  instructional  materials,  evaluating  development  tools,  and  defining  the  technical  issues  of  required 
resources.  Therefore  this  project  provides  them  a real  world  application  opportunity  in  terms  of  labor.  All  labors 
provide  students  an  insight  about  how  that  labors  are  conducted  in  real  life.  Also  labors  of  other  part  of  community 
provide  them  to  work  under  a project  leader,  evaluating  feedback.  Rules  that  defined  by  groups  also  indicated  labors 
of  each  group  members  as  well  as  rules  for  risk  management  and  work  policy.  Thus  students  experienced  some 
skills  related  team  work,  time  management  and  risk  management.  Shortly  game  development  has  lots  of  dynamics 
and  each  dynamics  has  a potential  of  some  skills  for  novice  instructional  designers. 

Conclusion 

Game  design  process  has  several  interaction  and  some  challenges  for  novice  instructional  designers.  This  kind  of 
projects  can  be  used  to  educate  instructional  designers  in  an  almost  real-life  experience.  Students  can  acquire 
practical  skills  which  they  can  use  in  real  life  because  they  faced  with  real  life  problems  in  this  process.  In  ID  field 
there  are  several  instructional  models  have  being  developed,  however  there  is  a lack  of  comparative  or  evaluative 
studies  for  these  models  ( Akilli  & Cagiltay,  2006).  Results  of  this  study  might  provide  an  assessment  system  with  by 
means  of  revealing  interaction  patterns  between  several  factors  in  game  design  activity.  These  outcomes  can  provide 
a guide  for  developing  new  project  based  courses  aiming  development  of  educational  games  or  other  kind  of 
multimedia  products. 

Observing  the  case  in  an  activity  approach  provide  an  in-depth  evaluation  of  processes.  Especially  in  group  work  it 
is  difficult  to  evaluate  students  individually.  Limitations  of  activity  system  might  reduce  the  quality  of  work.  When 
the  whole  system  observed,  instructors  can  make  fair  evaluation.  Also  outcomes  of  activity  should  be  observed  to 
understand  the  objects  of  the  course  is  not  the  unique  outcomes  of  the  system.  Activity  system  might  be  a good 
approach  for  evaluation  of  students  who  are  treated  with  constructivist  approaches  which  have  gaps  in  evaluation 
methods  (Airasian  & Walsh,  1997). 
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Measuring  College  Students’  Perceptions  and  Attitudes  toward  Anti- 
Plagiarism  Detection  Tools  and  Their  Behaviors,  Beliefs,  and  Moral 
Judgment  regarding  Plagiarism 

Aaron  Kim  and  Jean-Marc  Wise 
Florida  State  University 

Abstract 

The  focus  of  this  study  is  on  measuring  the  impact  of  using  anti -plagiarism  tools  (APT’s)  on  students’  behavior  and 
attitudes  toward  plagiarism.  A survey  instrument  was  developed  and  validated  in  order  to  collect  self-reported  data 
on  perceptions/attitudes,  behavior,  beliefs,  and  moral  judgment  with  respect  to  plagiarism  and  the  use  of  APT’s.  The 
survey  was  deployed  with  an  undergraduate  section  at  a southeastern  research  I university,  yielding  a response  rate 
of  72.3%  (n=133).  Results  found  significant  correlations  between  APT  perception  and  plagiarism  behavior,  moral 
judgment,  and  neutralization  (i.e.  justification  of  unethical  behavior).  Findings  also  confirmed  previous  research 
with  respect  to  correlations  between  attitudes,  moral  judgment,  and  plagiarism  behavior.  Details  on  survey 
development  and  validity,  as  well  as  suggestions  for  future  research  are  provided. 

Literature  Review 

It  is  fair  to  say  that  plagiarism  may  threaten  the  integrity  of  higher  education  in  colleges  and  universities 
throughout  the  U.S.  About  one  in  two  students  have  cheated  on  tests  and  71%  and  85%  of  college  students  who 
participated  in  a study  reported  plagiarism  and  cheating  on  assignments,  respectively  (Stephens  et  al,  2007). 
Although  Stephens  defines  cheating  and  plagiarism  rather  widely,  figures  from  other  studies  are  also  staggering.  For 
example,  70%  of  18,000  high  school  students  admitted  they  had  cheated;  and  60%  plagiarized  (Villano,  2006). 

Even  more  alarmingly,  most  of  the  students  reported  in  Villano’ s article  do  not  consider  cheating  as  wrong  and 
they  feel  justified  in  their  behavior.  “They  don’t  understand  why  they  should  be  held  to  a higher  standard”  said 
McCabe,  who  has  studied  the  field  of  academic  integrity  for  decades,  in  an  interview  with  Slobogin  from  CNN 
(Slobogin  2002).  The  report  is  consistent  with  the  findings  from  Stephens’  study  in  which  the  students  who  reported 
cheating  and  plagiarizing  showed  a lower  sense  of  moral  responsibility  and  a higher  tendency  of  neutralization  or 
justification  of  their  behaviors  than  non-cheating  students  (Stephens  2001).  Students’  behaviors  and  moral  sense 
have  been  further  investigated  by  Lovett-Hooper  et  al.  They  looked  at  the  relationship  between  academic  dishonesty 
and  norm/rule-violating  behaviors  (Lovett-Hooper  et  al.,  2007).  The  study  found  a positive  relationship  between  all 
three  Academic  Dishonesty  subscales  (self-dishonest,  social  falsifying,  and  plagiarism)  and  an  imagined  norm/rule- 
violating  future.  In  short,  the  study  suggests  academic  dishonesty  may  lead  to  rule-violating  behavior  in  the  future. 

In  2002,  the  Association  of  American  Colleges  and  Universities  (AAC&U)  published  a report  on  higher 
education  of  the  21st  century  to  suggest  comprehensive  principles  and  models  that  improve  learning  for  all 
undergraduate  students  entitled  “Greater  Expectations:  A New  Vision  for  Learning  as  a Nation  Goes  to  College” 
(AAC&U,  2002).  It  envisions  what  society  expects  from  college  education  and  identifies  three  characteristics  of 
learners  as  being  empowered,  informed,  and  responsible. 

Among  dishonest  behaviors,  plagiarism  has  become  a focal  point  recently  as  information  technology  advances 
and  the  use  of  the  Internet  is  commonplace.  In  surveys  run  from  2002  through  2005,  38%  of  18,000  college-aged 
students  said  they  had  ‘cut-and-pasted’  from  the  Internet,  25%  of  graduate  students  also  reported  engaging  in 
plagiarism,  and  80%  faculty  observed  plagiarism  (McCabe,  2005). 

In  an  effort  to  fight  plagiarism,  many  institutions  put  in  place  anti-  plagiarism  tools  (APT)  such  as  an  online 
plagiarism  detection  service.  There  exist  several  commercial  services  including  EVE2  (Easy  Verification  Engine: 
http://www.  canexus.com/eve/),  Glatt  Plagiarism  Services  (http://www.  plagiarism.com/),  SafeAssignment 
(http://www.mydropbox.com  ),  and  Turnltln  (htpp://www. turnitin.com).  These  tools  are  generally  well  received  by 
students  because  of  the  convenience  of  online  submission,  and  by  instructors  as  an  additional  tool  to  address  the 
plagiarism  issue.  But  only  few  studies  have  evaluated  such  tools  systematically.  In  a 2006  study,  Turnltln  was 
evaluated  in  terms  of  how  the  system  works,  user  experiences,  and  data  generated  by  the  system  (Evans,  2006).  The 
author  suggestes  that  the  service  detects  problematic  practices  but  it  may  not  be  practical  to  check  every  script 
thoroughly.  The  verdict  was  ‘working’  but  ‘impractical.’ 

The  growing  popularity  of  these  tools  is  likely  due  to  a variety  of  reasons.  For  one,  there  seems  to  be  a general 
agreement  among  instructors  that  awareness  of  such  a tool  can  be  a deterring  factor;  students  do  not  risk  getting 
caught  if  such  a tool  is  in  place.  As  shown  in  a recent  study  by  Dahl  (Dahl,  2007)  one  of  the  main  advantages  of 
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APT’s  may  be  their  psychological  effect.  Awareness  of  Turnltln  among  the  participants  in  the  study  increased 
awareness  of  academic  integrity  and  plagiarism  (e.g.,  what  is  considered  plagiarism,  honor  code,  etc).  The 
awareness  also  clearly  brought  about  some  behavioral  changes:  More  than  half  of  the  participants  learned  how  to 
reference  correctly  after  using  Tunrltln,  which  tells  us  that  such  a tool  can  also  have  educational  benefits  for  higher 
academic  standard  (Dahl,  2007). 

Therefore,  it  is  important  to  examine  how  using  APTs  can  affect  students’  behaviors,  attitudes,  and  perceptions 
with  respect  to  ethical  conduct.  What  psychological  effects  and  educational  benefits  of  using  APT's  can  bring  is  an 
important  question.  Thus,  in  this  current  study  we  investigate  the  following  questions: 

1 . What  are  college  students’  awareness,  perceptions  and  attitudes,  and  beliefs  with  respect  to  anti-plagiarism 
tools? 

2.  What  is  the  relationship  between  perceptions/attitudes  toward  plagiarism,  plagiarism  behavior,  moral 
judgment  and  beliefs,  and  perceptions/attitudes  toward  anti-plagiarism  tools? 

3.  What  are  the  effects  of  using  anti -plagiarism  tools? 

4.  What  is  the  relationship  between  effects  of  using  APT's  and  other  variables  of  interest? 

To  answer  the  research  questions,  a self-report  questionnaire  was  developed  and  used  to  collect  data. 
Correlational  analysis  was  the  main  method  used  to  examine  the  relationships  among  variables  based  on  the  students 
ratings  on  the  corresponding  statements.  The  questionnaire  included  statements  related  to  specific  effects  and 
benefits  of  APT's  suggested  in  the  literature  as  an  attempt  to  address  the  questions.  The  statements  include  ‘After  I 
learned  my  instructor  was  using  an  APT,  I still  plagiarized'  and  ‘I  decided  not  to  plagiarize’  (deterrence  effect);  ‘I 
became  more  aware  of  plagiarism’  (awareness  effect);  ‘I  tried  to  find  out  how  to  avoid  committing  plagiarism,’  ‘I 
ttied  to  learn  how  I should  reference  something  correctly  because  I experienced  an  APT'  and  ‘I  feel  confident  about 
quoting  other  people’s  work  correctly’  (educational  benefit).  Other  variables’  operational  definitions  are  as  follows. 

Variables  of  interest 

Plagiarism  awareness  is  the  level  of  students’  awareness  or  knowledge  about  what  constitutes  plagiarism. 
Plagiarism  behavior  represents  how  many  times  a student  engaged  in  plagiarism.  Perception/ attitude  on  plagiarism 
includes  three  subsets:  moral  judgment,  belief  (neutralization),  and  plagiarism  perception.  In  a 1991  study,  moral 
judgment  was  shown  to  have  a strong  relationship  with  college  students’  dishonest  behavior  (i.e.,  cheating  on  exam) 
and  suggested  a strong  predictor  of  such  behaviors  (Beck  & Ajzen,  1991).  The  study  suggested  that  students  with 
high  moral  judgment  level  are  less  likely  to  engage  in  dishonest  acts.  In  another  study,  neutralization  was  also 
shown  as  a strong  predictor  of  academic  dishonesty  (Diekoff  et  al.,  1996).  Neutralization  means  students’  tendency 
to  justify  dishonest  behaviors  attributing  external  and  internal  factors.  Thus,  the  study  suggested  that  students  with 
high  neutralization  level  are  more  likely  to  engage  in  dishonest  acts.  Perception/attitude  of  APT’s  is  about  how 
students  feel  about  APT’s,  whether  they  support  the  use  of  APT's,  feel  intimidated,  see  APT's  as  a reliable  tool,  and 
so  forth. 

Method 

Participants 

Participants  were  184  students  enrolled  in  a non-major  course  in  southeastern  public  university.  The 
researchers  decided  to  use  one  course  that  is  representative  of  the  university  population  and  work  with  the  instructor 
of  the  course  to  ensure  reasonable  response  rate.  The  instructor  of  the  course  gave  extra  credit  to  the  students  who 
completed  the  survey.  The  response  rate  was  72.3%  (133  students  participated).  Since  the  survey  should  be 
anonymous,  the  researcher  assured  the  students  that  the  fact  of  their  participation  would  be  stored  separately  from 
the  survey  data  and  that  their  responses  would  not  be  associated  with  their  identity. 

The  respondents  consisted  of  6 freshmen  (4.5%),  55  sophomores  (41.4%),  56  juniors  (42.1%),  and  15  seniors 
( 1 1.3%).  64%  of  the  respondents  were  female.  The  majority  of  the  students  major  in  Communications  Studies  (42%) 
and  other  majors  included  English,  Marketing,  Media  Production,  Business,  Public  Relations,  Political  Science,  and 
International  Affairs. 

Procedures 

The  first  announcement  regarding  the  study  and  the  procedure  was  made  during  the  class  in  September  2009 
and  was  followed  by  an  email  advertisement  and  announcement  on  the  course  website.  All  students  enrolled  in  the 
course  were  emailed  with  a link  to  the  online  survey  along  with  a brief  summary  about  the  study.  Anonymity  of 
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participation  was  emphasized  in  all  announcements  and  individual  inquiries  regarding  anonymity  and  procedures 
were  answered  during  the  class  visit  and  via  emails  by  the  researchers  and  the  instructor.  Once  the  students  opened 
the  questionnaire,  they  needed  to  provide  their  informed  consent  in  order  to  proceed. 

The  online  questionnaire  was  available  for  one  week.  In  addition  to  the  announcements  in  class  and  on  the 
course  site,  and  the  initial  email  invitation,  a reminder  email  was  sent  out  on  the  third  day  and  another  one  on  fifth 
day  of  the  data  collection. 

Measures 

A survey  instrument  was  developed  for  this  study,  which  included  newly  developed  items  and  other  items 
that  were  adapted  from  the  existing  instruments.  It  is  a self-report  questionnaire  that  consists  of  forty  five  statements, 
most  of  which  are  rated  using  5-point  Likert  scale  ranging  from  1 (strongly  disagree)  to  5 (strongly  agree),  and 
eleven  statements  with  slightly  different  scales.  Please  see  Appendix  A for  a copy  of  the  instrument.  New 
statements  were  written  in  order  to  measure  the  variables  of  interest:  the  students’  awareness,  perceptions  and 
attitudes,  beliefs,  and  morality  with  respect  to  plagiarism  and  APT’s.  Subscales  of  moral  judgment  and  beliefs  were 
adapted  from  a moral  responsibility  scale  (Beck  & Ajzen,  1991)  and  a neutralization  scale  (Diekoff  et  ah,  1996). 
Students’  academic  dishonesty  was  measured  by  items  adapted  from  the  Academic  Dishonesty  Student  Survey 
(McCabe,  1992). 

Structure  of  the  measurement  instrument 

The  measurement  instrument  consists  of  5 sections  measuring  the  variables  of  interest  in  addition  to  APT 
experience  (section  4)  and  the  demographic  information  (section  7).  The  following  Table  1 summarizes  the 
structure  of  the  instrument  with  the  different  sections  and  associated  variables. 


Table  1.  Structure  of  the  Measurement  Instrument 


Section  Variables 

Subscale  (if  any) 

Scale 

Variable  type 

1 . Plagiarism  Awareness 

Disagree,  Not  sure,  and 

Interval  (Pretest 

Agree. 

total  score) 

2.  Plagiarism  Behavior 

Never,  0,  to  More  than  three 
times,  5 

Interval 

3.  Perception/Attitude  toward 

- Moral  judgment 

5 -point  Likert  scale  ranging 

Interval 

Plagiarism 

- Neutralization 

from  1 (strongly  disagree)  to 

- Plagiarism  perception 

5 (strongly  agree). 

4.  Anti-Plagiarism  Tool  (APT) 

- APT  Awareness 

- Yes,  No 

- Categorical 

Experience 

- Experience 

- Number  of  courses  and 
instructors 

- Interval 

5.  Perception/attitude  of  Anti- 

5 -point  Likert  scale  ranging 

Interval 

Plagiarism  Tool  (APT) 

from  1 (strongly  disagree)  to 

6.  Effects  of  APT 

- Educational  Benefits 

5 (strongly  agree)  with  N/A 
option 

Interval 

7.  Demographic  Information 

Categorical 

Section  2 uses  a modified  version  of  McCabe's  Academic  Dishonesty  Student  Survey  (1992).  Most 
statements  in  the  section  3 were  developed  in  accordance  with  items  in  sections  5 and  6.  Section  3, 
perception/attitude  on  plagiarism,  contains  subscales  of  moral  judgment  and  beliefs  that  were  adapted  from  a moral 
responsibility  scale  (Beck  and  Ajzen,  1991)  and  neutralization  scale  (Diekoff  et  al.  1996). 

Development  of  the  measurement  instrument 

Newly  developed  items  were  drawn  from  the  literature  and  tested  and  revised  through  a panel  review,  an 
expert  review,  and  formative  evaluation.  The  instrument  with  the  new  items  was  reviewed  by  twelve  doctoral 
students  who  are  experienced  in  survey  research  and  instrument  development.  The  revised  instrument  then  was 
reviewed  by  an  expert  who  has  been  conducting  and  teaching  survey  research  methods  at  a Research  I University  for 
more  than  20  years. 

Formative  evaluation  involved  four  representative  students  with  different  majors  and  years  in  school.  During 
the  formative  evaluation  session  students  engaged  in  open  discussion  and  critique  of  the  survey  instrument  (e.g., 
clarity  of  statements,  overall  structure,  effective  communication,  etc).  The  instrument  was  once  again  revised  based 
on  the  result  of  the  formative  evaluation. 
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Validity  and  reliability  test 

Formative  evaluation  provided  opportunities  for  assessing  not  only  the  quality  of  items  but  also  content 
validity.  Since  there  is  no  external  criterion  that  can  be  compared  to  in  terms  of  plagiarism  behavior,  perception,  and 
morality,  criterion  validation  is  beyond  the  scope  of  this  study.  Establishing  construct  validity,  however,  may  be 
possible  for  some  items  that  were  modified  from  the  existing  items.  The  authors  made  a judgment  based  on  how  the 
results  of  this  study  relate  to  the  previous  studies  that  used  the  same  instrument.  For  example,  the  instrument 
measuring  moral  judgment  and  beliefs  revealed  that  plagiarism  is  highly  correlated  with  moral  judgment  and  beliefs 
of  students  (Beck  & Ajzen,  1991).  Thus,  construct  validity  of  the  modified  items  of  moral  judgment  and  belief 
subscales  can  be  assessed  in  addition  to  the  content  validity. 

The  reliability  estimates  for  each  section  and  subscales  are  summarized  in  Table  2. 


Table  2.  Reliability  Estimates 


Section  Variables 

N of  Items 

Cronbach's  Alpha 

1 . Plagiarism  Awareness 

6 

N/A  - a test  score 

2.  Plagiarism  Behavior 

9 

.763 

3.  Perception/attitude  on  Plagiarism 

11 

.693 

a.  - Moral  judgment 

(4)* 

(.660) 

b.  - Neutralization 

(2) 

(.440) 

c.  - Plagiarism  perception 

(5) 

(.737) 

4.  Anti -Plagiarism  Tool  (APT)  Experiences 

5 

N/A** 

5.  Perception/attitude  of  Anti -Plagiarism 
Tool  (APT) 

13 

.701 

6.  Effects  of  APT 

7 

.727 

7.  Demographic  Information 

6 

N/A 

*subscales  in  parenthesis 

**2  questions  pertains  to  the  number  of  courses  and  instructors  the  students  counted  as  experiences  with  APT  and 
plagiarism  discussion. 


Results 

College  students’  awareness,  perceptions  and  attitudes,  and  beliefs  with  respect  to  anti-plagiarism  tools 

The  awareness  level  was  measured  by  six  questions  each  asking  whether  a specific  act  was  a type  of 
plagiarism.  The  majority  of  the  students  scored  3 or  less,  which  may  indicate  the  lack  of  knowledge  about 
plagiarism  among  the  students.  Moral  judgment,  plagiarism  perception,  neutralization,  and  APT  perception  are  all 
quite  high.  APT  effects  showed  high  agreement  that  their  experiences  with  APT  affected  their  behaviors  and 
perception.  The  following  descriptive  statistics  table  summarizes  the  results. 


Table  3.  Descriptive  Statistics 


N 

Range 

Min  Max 

Mean 

Std. 

Deviation 

Skewness 
Statistic  Std. 

Error 

Plagiarism 

Awareness 

124 

.00 

6.00 

2.137 

.990 

.383 

.217 

Moral  Judgment 

123 

6.00 

15.00 

11.894 

2.003 

-.350 

.218 

Plagiarism 

Perception 

123 

9.00 

25.00 

20.667 

2.748 

-.750 

.218 

Neutralization 

122 

2.00 

8.00 

3.943 

1.501 

.412 

.219 

APT  Perception 

124 

24.00 

59.00 

40.742 

7.204 

-.042 

.217 

Data  Preparation  for  statistical  analysis 

Preliminary  data  examination  showed  that  there  were  some  missing  cases  and  undifferentiated  responses. 
Further  examination  of  such  cases  convinced  the  authors  to  drop  the  undifferentiated  cases  from  the  analysis.  A total 
of  10  cases  was  removed  from  the  data  set  because  they  either  showed  no  variance  in  their  answers  to  one  or  more 
sections  of  the  questionnaire  or  provided  answers  that  did  not  match  when  they  were  supposed  to  match  such  as 
reverse  coded  items. 

In  order  to  address  the  issue  of  missing  data,  we  calculated  the  percent  score  for  some  variables  instead  of  raw 
score  using  only  answered  items.  Since  there  were  only  a few  missing  values  and  the  missing  values  should  not 
affect  the  overall  score,  a raw  score  divided  by  the  maximum  possible  score  would  best  represent  the  measures. 
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Perception/attitude  on  plagiarism  (moral  judgment,  belief  (neutralization),  APT  effects,  and  plagiarism  perception 
used  percentage  scores  while  plagiarism  awareness  and  behavior  used  raw  scores. 

Correlation  Analysis 

This  study  examined  the  correlations  among  moral  judgment,  beliefs  (in  the  neutralization  subscale), 
plagiarism  behaviors,  and  attitudes/perceptions  toward  anti-plagiarism  tools  (APT’s).  The  following  correlational 
analysis  table  summarizes  the  results  of  this  study.  It  shows  relationships  among  variables  and  the  effects  of  APT 
use.  Educational  benefit  is  a subscale  under  APT  effects  that  included  items  such  as  ‘I  became  more  aware  of 
plagiarism’ ; ‘I  tried  to  find  out  how  to  avoid  committing  plagiarism’ ; ‘I  would  prefer  to  get  more  guidance  about 
what  plagiarism  is’;  ‘I  tried  to  learn  how  I should  reference  something  correctly  because  I experienced  an  APT.’ 


Table  3.  Correlations 

Awrns 

Plagiaris 

m 

Moral 

Judgment 

Plagiarism 

Perception 

Neutralizatio 

n 

APT 

Perception 

Awareness  1 

.036 

.177 

-.007 

.011 

.058 

Plagiarism  Behavior 

1 

-.339(**) 

-.026 

.305(**) 

-.310(**) 

Moral  Judgment 

1 

.209(*) 

-.429(**) 

,307(**) 

Plagiarism  Perception 

1 

-.206(*) 

.158 

Neutralization 

1 

-.382(**) 

APT  Perception  1 

APT  Effects 
Ed.  benefit 
Experience 

**  Correlation  is  significant  at  the  0.01  level  (2 -tailed). 

* Correlation  is  significant  at  the  0.05  level  (2-tailed). 

Moral  judgment  and  perception  of  APT  are  negatively  correlated  with  plagiarism  behavior  while 
neutralization  is  positively  correlated.  Plagiarism  behaviors  represent  the  frequency  of  dishonest  behaviors  that  the 
students  reported.  Thus,  the  study  confirmed  that  the  higher  their  moral  judgment,  the  less  likely  it  is  for  the  students 
to  involve  in  plagiarism  activities.  Likewise,  the  more  the  students  have  a positive  perception  of  APT's,  the  less 
likely  to  plagiarize.  On  the  other  hand,  the  more  students  tend  to  justify  their  dishonest  behavior,  the  more  likely 
they  are  to  plagiarize. 

Perception  of  APT  correlates  positively  with  moral  judgment  and  correlates  negatively  with  neutralization, 
which  could  mean  that  those  students  who  perceive  plagiarism  negatively  tend  to  avoid  plagiarism  and  perceive 
APT's  positively.  APT  perception  is  also  positively  correlated  with  APT  effects.  In  other  words,  those  who  see 
APT's  positively  tend  to  attribute  their  specific  perception  and  behavioral  changes  to  their  experience  with  APT's, 
leading  to  a higher  plagiarism  awareness  level  or  a decision  not  to  plagiarize. 

APT  effects 

Based  on  the  students’  responses,  the  majority  of  the  students  agreed  that  APT  use  affected  their  behavior 
and  awareness  with  respect  to  plagiarism.  Table  4 summarizes  the  students’  responses  to  the  APT  effects  statements 
and  Table  5 summarizes  the  correlation  between  APT  effects  and  other  variables. 


Table  4.  APT  Effects 


N/A 

Strongly 

disagree 

Disagree 

Neutral 

Agree 

Strongly 

agree 

Total 

1 

25 

64 

28 

6 

123 

20.3% 

52.0% 

22.8% 

4.9% 

2 

6 

5 

11 

19 

68 

12 

123 

4.9% 

4.1% 

8.9% 

15.4% 

55.3% 

9.8% 

3 

30 

5 

5 

23 

43 

16 

123 

24.4% 

4.1% 

4.1% 

18.7% 

35.0% 

13.0% 

4 

15 

11 

10 

29 

50 

8 

123 

260 


5 


123 


12.2% 

8.9% 

8.1% 

23.6% 

40.7% 

6.5% 

2 

16 

41 

43 

20 

1 

1.6% 

13.0% 

33.3% 

35.0% 

16.3% 

0.8% 

1 

2 

7 

18 

80 

15 

0.8% 

1.6% 

5.7% 

14.6% 

65.0% 

12.2% 

16 

8 

22 

27 

42 

6 

13.0% 

6.5% 

17.9% 

22.0% 

34.1% 

4.9% 

*1. 1 still  plagiarized;  2. 1 became  more  aware  of  plagiarism;  3. 1 decided  not  to  plagiarize;  4. 1 tried  to  find  out  how 
to  avoid  committing  plagiarism;  5. 1 would  prefer  to  get  more  guidance  about  what  plagiarism  is;  6. 1 feel  confident 
about  quoting  other  people’s  work  correctly;  7.1  tried  to  learn  how  I should  reference  something  correctly  because  I 
experienced  an  APT.  APT  effects  are  deterrence  effect  (statement  1 and  3), 
awareness  effect  (statement  2),  and  educational  benefit  (statement  4,  5,  6,  and  7) 


Table  5.  Correlation  Analysis  with  APT  Effects 


Awareness 

Plagiaris 

m 

Behavior 

Moral 

Judgment 

Plagiarism 

Perception 

Neutralization 

APT 

Perception 

Awareness 

1 

.036 

.209(*) 

-.007 

.011 

.081 

Plagiarism 

Behavior 

.036 

1 

-.278(**) 

-.026 

.305(**) 

-.293(**) 

Moral 

Judgment 

.209(*) 

-,278(**) 

1 

.151 

-.372(**) 

.313(**) 

Plagiarism 

Perception 

-.007 

-.026 

.151 

1 

-.206(*) 

.128 

Neutralization 

.011 

.305(**) 

-.372(**) 

-.206(*) 

1 

-,325(**) 

APT 

Perception 

.081 

-.293(**) 

.313(**) 

.128 

-.325(**) 

1 

APT  Effect  1* 

.036 

.352(**) 

-.132 

-.071 

.071 

-.187 

2 

.183(*) 

.050 

.197(*) 

.036 

-.069 

.070 

3 

.076 

.049 

.152 

.173 

-.078 

.138 

4 

.033 

.039 

.167 

-.018 

-.065 

.054 

5 

-.022 

.108 

.004 

-.010 

.010 

.020 

6 

.045 

-.236(**) 

.212(*) 

.097 

-.411(**) 

,377(**) 

7 

.162 

-.089 

.060 

.092 

-.118 

.210(*) 

* Analyzed  APT  effects  that  did  not  include  n/a  responses. 

Students’  plagiarism  behavior  was  directly  reflected  on  the  first  statement  of  APT  effects  and  the 
correlation  between  plagiarism  behavior  and  APT's  deterrence  effect  was  positive  and  significant.  Likewise, 
students’  awareness  level  was  positively  correlated  with  APT's  awareness  effect  item.  Other  significant  correlations 
were  found  between  one  of  the  APT's  educational  benefit  item  (‘I  feel  confident  about  quoting  other  people’s  work 
correctly')  and  plagiarism  behavior,  moral  judgment,  neutralization,  and  APT  perception/attitude,  respectively. 
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Discussion 


The  research  questions  of  this  study  were: 

1 . What  are  college  students’  awareness,  perceptions  and  attitudes,  and  beliefs  with  respect  to  anti-plagiarism 
tools? 

2.  What  is  the  relationship  between  perceptions/attitudes  toward  plagiarism,  plagiarism  behavior,  moral 
judgment  and  beliefs,  and  perceptions/attitudes  toward  anti-plagiarism  tools? 

3.  What  are  the  effects  of  using  anti-plagiarism  tools? 

4.  What  is  the  relationship  between  effects  of  using  APT's  and  other  variables  of  interest? 

With  the  first  two  questions  we  aimed  at  gaining  a better  understanding  of  the  factors  that  contribute  to  plagiarism 
behavior  in  students.  As  presented  in  the  results  section,  there  is  positive  relationship  between  moral  judgment,  and 
APT  perception/attitude.  On  the  other  hand,  neutralization  and  plagiarism  behaviors  have  a negative  relationship. 

The  third  and  forth  questions,  however,  could  not  be  answered  as  expected  since  there  is  not  enough 
information  available,  even  though  some  correlations  turned  out  significant  (i.e.,  moral  judgment  and  overall  APT 
effect  score,  r = .184,  p < .05;  APT  perception  and  APT  effect  score,  r = .253,  p < .01).  For  example,  educational 
and  other  benefits  of  using  APT’s  was  a main  variable  of  interest  in  this  study  but  could  not  be  tested  appropriately 
given  the  data  since  there  were  only  8 students  who  have  never  experienced  an  APT  before  whereas  the  rest  of  the 
students  have  experienced  it.  Although  correlations  are  statistically  significant,  this  may  not  be  a meaningful  result 
because  the  data  set  violates  the  normality  and  equal  variance  assumptions  that  need  to  be  met  to  run  a statistical 
test.  Since  most  students  already  knew  about  APT's  and  have  used  at  least  one  before,  there  must  be  other  ways  to 
differentiate  their  experience  with  APT's  or  a large  enough  number  of  non-experienced  users  should  be  surveyed. 

College  students’  awareness,  perceptions  and  attitudes,  and  APT  perception/attitude 

In  discussing  plagiarism  awareness,  perceptions  and  attitudes,  and  APT  perception  and  the  corresponding 
analysis,  it  is  important  to  note  that  most  of  the  students  who  participated  in  the  study  knew  about  APT’s  and  had 
experienced  APT's  before.  Only  eight  students  indicated  that  they  didn't  know  about  APT's  and  had  never  taken  a 
course  that  used  APT's.  Interestingly,  the  students’  awareness  level  turned  out  low  although  their  self-reported 
perception  of  plagiarism  and  moral  judgment  level  were  high.  This  may  suggest  that  they  don't  know  what 
constitutes  plagiarism  but  they  are  aware  that  plagiarism  is  an  issue  and  can  have  serious  consequences.  The  absence 
of  an  association  between  behavior  and  awareness  was  confirmed  by  analysis  of  correlation.  Also  plagiarism 
awareness  is  not  associated  with  moral  judgment,  plagiarism  perception,  neutralization,  or  APT  perception,  which 
makes  sense  because  knowing  about  plagiarism  does  not  necessarily  mean  high  level  of  moral  judgment  and  other 
variables. 

APT  perception/attitude  was  measured  in  this  study  along  with  other  variables  that  were  found  in  other 
academic  dishonesty  studies  in  order  to  examine  the  relationship  with  those  variables  (e.g.,  moral  judgment  and 
dishonest  behaviors).  APT  perception/attitude  was  found  to  have  a positive  relationship  with  moral  judgment  while 
having  a negative  relationship  with  plagiarism  behavior  and  neutralization.  It  seems  valid  and  reasonable  to  include 
APT  perception/attitude  as  a main  variable  in  plagiarism  research. 

This  study  shows  results  that  are  consistent  with  previous  studies  in  addition  to  presenting  new  findings  from 
including  additional  variables  such  as  APT  perception/attitude.  For  example,  the  measure  of  moral  judgment  and 
beliefs  confirmed  that  plagiarism  behavior  is  highly  correlated  with  moral  judgment  and  beliefs  of  students  (Beck  & 
Ajzen,  1991).  In  terms  of  validity  of  the  instrument  developed  for  this  study,  consistent  results  and  correlations 
among  major  variables  strongly  support  the  construct’s  validity  with  respect  to  the  modified  items  as  well  as  the 
newly  written  items. 

In  addition  to  confirming  findings  of  prior  research,  the  present  study  shows  that  the  use  of  APT’s  is  related 
to  how  students  behave  and  how  they  perceive  plagiarism.  The  fact  that  this  study  found  significant  relationships 
between  the  use  of  APT's  and  other  major  variables,  such  as  moral  judgment,  suggests  that  APT's  should  be 
included  as  a major  variable  in  future  research  on  plagiarism. 

APT  effects 

First,  as  to  the  deterrence  effect  (statement  1 and  3),  none  of  the  respondents  stated  that  they  plagiarized 
and  about  50%  of  them  indicated  they  decided  not  to  plagiarize  after  they  learned  that  APT's  were  used.  It  is 
particularly  important  to  note  that  about  20%  said  that  the  deterrence  effect  was  not  applicable  to  them.  During  the 
formative  evaluation,  all  of  the  students  pointed  out  that  the  ‘not  applicable’  option  was  needed  because  they  did  not 
plagiarize  and  decided  not  to  do  so  regardless  of  APT  use.  It  may  not  be  reasonable  to  make  a conclusive  remark 
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about  ‘not  applicable’  but  helpful  to  understand  and  the  difference  between  ‘agree’  and  ‘disagree’  responses.  In 
other  words,  by  excluding  ‘not  applicable’  responses,  agreement  with  the  deterrence  effects  of  APT  can  become 
clearer. 

The  awareness  effect  (statement  2,  ‘I  became  more  aware  of  plagiarism’)  was  also  clearly  shown  to  be  one 
of  the  main  APT  effects.  Using  an  APT  raised  the  level  of  plagiarism  awareness,  which  was  also  consistent  with 
students’  high  plagiarism  perception  score  (mean  = 20.67,  sd  = 2.75,  max  = 25).  The  APT  awareness  effect  is  also 
supported  by  the  correlations  with  plagiarism  awareness  and  moral  judgment  although  the  size  of  the  correlations  is 
quite  small.  On  the  other  hand,  plagiarism  perception  and  APT  awareness  effect  are  not  associated  based  on  the 
correlation  analysis.  Plagiarism  behavior  and  APT  perception  are  not  associated  either  with  APT  awareness  effect  as 
it  is  not  associated  with  plagiarism  awareness. 

Most  educational  benefit  items  also  did  not  produce  significant  correlations  with  the  other  variables.  Only 
the  responses  to  the  statement  6 (‘I  feel  confident  about  quoting  other  people’s  work  correctly’)  turned  out  to  have 
significant  correlations  with  plagiarism  behavior,  moral  judgment,  neutralization,  and  APT  perception.  About  77% 
of  the  respondents  agreed  with  the  statement,  which  may  indicate  benefit  of  using  APT’s.  In  fact,  many  of  the 
respondents  agreed  with  other  educational  benefit  statements  except  the  statement  5,  which  was  about  whether  they 
wanted  more  guidance  or  not.  If  they  were  already  confident  as  indicated  in  their  response  to  the  statement  6,  they 
would  not  seek  more  help.  In  their  responses  to  statement  4 and  7,  they  indicated  they  tried  to  leant  more  about 
avoiding  plagiarism  (about  50%  of  the  respondents  agreed  excluding  ‘n/a’  responses).  Despite  the  absence  of 
significant  correlation,  educational  benefits  seemed  to  be  clearly  shown. 

Limitation  of  the  Study 

This  study  involved  the  collection  of  sensitive  information  from  students  by  means  of  an  anonymous  web- 
based  survey.  In  order  to  encourage  participation,  the  survey  was  deployed  in  the  context  of  an  undergraduate  course 
and  students  were  able  to  earn  participation  points  toward  their  grade  as  a reward  for  participating  in  the  study. 
Although  the  students’  anonymity  was  guaranteed,  it  is  possible  that  some  students  were  reluctant  to  admit  to  any 
unethical  behavior  they  may  have  engaged  in,  which  might  explain  the  comparatively  low  scores  on  that  variable  in 
contrast  to  other  studies  of  undergraduate  students.  One  might  argue  that  self-reports  should  be  considered  with 
caution  in  general,  because  participants  might  have  a distorted  perspective  of  their  own  behavior;  they  might  report 
how  they  think  they  should  behave  or  how  they  would  like  to  behave,  rather  than  how  they  actually  behave. 
Additionally,  the  vast  majority  of  the  participants  had  had  some  experience  with  ATP's  in  the  past;  this  prevented  us 
from  effectively  comparing  the  responses  between  experienced  and  inexperienced  students  and  as  a consequence  it 
was  not  possible  to  establish  any  differences  between  the  groups.  A larger  sample  size  would  likely  yield  a better 
distribution  and  thus  offer  additional  opportunities  for  analysis. 

Recommendations  for  Future  Research 

Current  study  only  surveyed  133  students,  which  is  an  intact  group  to  ensure  higher  response  rate  and  achieve 
good  representativeness.  Despite  the  high  response  rate,  the  intact  group  participation  unexpectedly  resulted  in  too 
small  number  of  students  who  had  not  experienced  APT  prior  to  the  data  collection.  It  is  crucial  to  have  enough 
number  of  non-experienced  participants  to  compare  variables  based  on  the  student’s  experiences  with  APT.  Thus, 
more  participation  would  be  desirable  in  a future  research.  Also  it  would  be  ideal  to  have  any  way  to  complement 
and  support  the  findings  from  self-report  questionnaire  method.  Observation  (e.g.,  examining  actual  writing 
assignments)  would  provide  invaluable  insights  and  potential  verification  of  students’  responses. 

Since  it  can  be  hypothesized  that  there  are  differences  in  the  variables  of  interest  depending  on  levels  of 
awareness  of  and  perception/attitude  toward  APT’s,  it  may  prove  beneficial  to  employ  MANOVA  in  order  to 
investigate  the  effects  of  awareness  and  interactions  among  key  variables  (e.g.,  morality,  awareness,  and  attitude). 
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Appendix  A.  Survey  Instrument 


Date: 


Survey  Instrument  - FE  Form  #1  (v  2.5) 


Please  indicate  your  agreement  with  the  following  statements  ( Agree/  Disagree/  Not  sure  ) 

• Copying  any  part  of  source  texts  without  crediting  sources  is  considered  plagiarism. 

• Plagiarism  includes  paraphrasing  source  texts  without  crediting  the  source. 

• As  long  as  there  is  proper  citation,  you  can  copy  someone  else’s  words  in  your  paper  without 
quotation  marks. 

• Using  someone  else’s  idea  in  your  paper  without  copying  any  words  may  not  require  proper 
citation. 

• Submitting  someone  else’s  paper  is  acceptable  if  your  friend  exclusively  wrote  it  for  you. 

• Completely  paraphrasing  source  texts  may  not  require  proper  citation. 


How  many  times  have  you  done  the  following  in  the  past  academic  year  (2008-2009)? 

( never  / once  / twice  / more  than  twice) 

• working  with  others  on  an  assignment  when  asked  for  individual  work. 

• copying  a few  sentences  from  an  internet  source  without  properly  citing  it. 

• turning  in  work  that  I have  partially  copied  from  another  person. 

• turning  in  an  assignment  that  I have  partially  copied  from  assignments  I previously  submitted. 

• paraphrasing  material  from  a written  source  without  citing  it. 

• receiving  unpermitted  help  from  someone  on  an  assignment. 

• falsifying  information  on  a bibliography. 

• turning  in  a paper  found/purchased  on  the  internet  as  my  own. 

• submitting  a paper  someone  else  wrote  for  me  as  my  own. 


Please  rate  the  following  statements. 

(Strongly  disagree  - 1,  Disagree  - 2,  Neutral  - 3,  Agree  - 4,  Strongly  agree  - 5) 

• Plagiarism  is  a serious  offense. 

• It  would  be  morally  wrong  for  me  to  cheat  on  an  exam. 

• Cheating  is  okay  if  everyone  else  seems  to  be  cheating. 

• Plagiarism  is  a moral  issue. 

• Plagiarism  is  a problem  at  FSU  that  needs  to  be  addressed. 

• I am  aware  of  the  consequences  in  regards  to  plagiarism. 

• I have  read  FSU's  Academic  Honor  Policy. 

• FSU  promotes  academic  integrity  in  every  way  possible. 

• Cheating  on  exams  goes  against  my  moral  principles. 

• Cheating  is  okay  if  a friend  asked  me  to  help  him/her  cheat. 

• I would  face  serious  consequences  if  I were  caught  plagiarizing. 

• I am  familiar  with  the  content  of  FSU's  Academic  Honor  Policy. 
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Are  you  aware  of  the  following? 

1.  software  or  online  services  that  detect  plagiarism  (anti-plagiarism  tools)  exist.  ( Yes  / No  ) 

2.  anti-plagiarism  tools  are  currently  available  to  FSU  instructors.  ( Yes  / No  ) 

3.  Some  FSU  instructors  currently  use  plagiarism  detection  tools.  ( Yes  / No  ) 

Have  you: 

1.  taken  at  least  one  course  that  has  used  an  anti-plagiarism  tool?  How  many? 

2.  had  any  instructors  who  talked  about  plagiarism  and  how  to  avoid  it  in  class?  How  many? 

(None,  1,  2,  3,  4,  5 or  more.) 


Anti-plagiarism  tools  (APT's)  are  commercial  online  services  that  check  various  sources  including 
student  paper  databases  to  detect  plagiarized  papers.  Currently,  FSU  has  two  successful  commercial 
APT's  available  for  use  by  the  FSU  faculty. 

Please  rate  the  following  statements. 

(Not  applicable  - 0,  Strongly  disagree  - 1,  Disagree  - 2,  Neutral  - 3,  Agree  - 4,  Strongly  agree  - 5) 

• I support  the  use  of  APT’s  for  undergraduates. 

• I am  concerned  about  writing  due  to  APT’s. 

• Using  APT’s  pose  no  risk  to  honest  students. 

• I am  worried  that  I could  get  caught 

• I feel  offended  by  the  use  of  APT's  because  it  feels  like  I am  being  suspected  of  plagiarizing. 

• I believe  that  trust  between  student  and  instructor  diminishes  as  a result  of  using  APT's. 

• No  APT’s  can  detect  all  cases  of  plagiarism. 

• APT's  are  deterrent  to  plagiarism. 

• APT's  are  intimidating  to  me. 

• I understand  how  APT’s  work. 

• I feel  uncomfortable  with  APT’s  because  they  may  wrongfully  accuse  me  of  plagiarism. 

• My  experience  with  APT's  has  been  a positive  one. 

• Overall,  it  seems  that  APT’s  are  reliable  tools. 


After  I learned  my  instructor  was  using  an  APT, 

• I still  plagiarized. 

• I became  more  aware  of  plagiarism. 

• 1 decided  not  to  plagiarize. 

• I tried  to  find  out  how  to  avoid  committing  plagiarism. 

• I tried  to  find  out  how  the  system  works. 

Please  rate  the  following  statements. 

• I would  prefer  to  get  more  guidance  about  what  plagiarism  is. 

• I feel  confident  about  quoting  other  people’s  work  correctly. 

• My  attitude  towards  plagiarism  has  changed  after  encountering  APT’s. 

• I tried  to  learn  how  I should  reference  something  correctly  because  I experienced  an  APT. 

• I have  used  a specific  technique  to  attempt  to  "trick"  the  tool. 
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(ALTERNATIVE  sections  for  those  who  have  never  experienced  APT’s) 

Anti -plagiarism  tools  (APT's)  are  commercial  online  services  that  check  various  sources  including  student 
paper  databases  to  detect  plagiarized  papers.  Such  tools  compare  submitted  text  with  millions  of  papers  in 
various  databases  and  billions  of  web  pages  to  identify  potential  plagiarized  text.  A report  (called 
originality  report)  is  generated  for  each  submitted  paper.  The  report  presents  overall  matching  percentage, 
the  submitted  text  with  potentially  plagiarized  portions  highlighted  and  original  sources  for  further 
investigation.  Currently,  FSU  has  two  successful  commercial  APT’s  available  for  use  by  the  FSU  faculty. 

Please  rate  the  following  statements. 

(Strongly  disagree  - 1,  Disagree  - 2,  Neutral  - 3,  Agree  - 4,  Strongly  agree  - 5) 

• I would  support  the  use  of  APT’s  at  FSU. 

• I would  be  concerned  about  writing  due  to  APT’s. 

• Using  APT’s  would  pose  no  risk  to  honest  students. 

• I would  be  worried  that  I could  get  caught. 

• I would  feel  offended  by  the  use  APT’ s because  it  would  feel  like  I was  being  suspected  of 
plagiarizing. 

• I believe  that  trust  between  student  and  instructor  would  diminish  as  a result  of  using  APT's. 

• No  APT’s  could  detect  all  cases  of  plagiarism. 

• Using  an  APT’s  would  be  a deterrent  to  plagiarism. 

• APT's  would  be  intimidating  to  me. 

• I would  feel  uncomfortable  with  APT’s  because  they  can  wrongfully  accuse  me  of  plagiarism. 


Please  rate  the  following  statements. 

(Strongly  disagree  - 1,  Disagree  - 2,  Neutral  - 3,  Agree  - 4,  Strongly  agree  - 5) 

If  I learned  my  instructor  was  using  an  APT, 

• I would  still  plagiarize. 

• I would  become  more  aware  of  plagiarism. 

• I would  decide  not  to  plagiarize. 

• I would  try  to  find  out  how  to  avoid  committing  plagiarism. 

• I would  try  to  find  out  how  the  system  works. 


Please  rate  the  following  statements. 

If  I experienced  APT’s  for  a while, 

• I would  prefer  to  get  more  guidance  about  what  plagiarism  is. 

• I would  feel  confident  about  quoting  other  people’s  works  correctly  after  learning  about  plagiarism 
and  academic  integrity. 

• My  attitude  towards  plagiarism  would  change. 

• I would  try  to  learn  how  I should  reference  something  correctly. 

• I would  try  to  find  specific  techniques  that  could  "trick"  the  tool. 
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Demographic  Information 

• What  is  your  age: 

• What  is  your  gender? 

0 Male 
0 Female 

• What  is  your  race? 

0 African  American/Black 
0 Asian 

0 Caucasian/White 
0 Hispanic 
0 Multi-racial 

0 Native  American/American  Indian 
0 I prefer  not  to  answer 
0 Other: 

• What  is  your  major? 

0 Major  list. 

• What  is  your  year  in  school 

O Freshman 
O Sophomore 
O Junior 
O Senior 
O Graduate 
O Other 

• What  is  your  overall  cumulative  GPA? 

0 1 .99  or  less 

0 2.0  - 2.49 

0 2.5-2.99 

0 3.0  - 3.49 

0 3.5 -4.0 
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Do  teachers  enrolled  in  an  online  Science  course  learn  more  when 
participating  in  discussion  forums? 


Todd  Kisicki,  Heidi  Blair,  and  Brian  Nelson 
Arizona  State  University 

Abstract 

This  study  examined  the  impact  of  discussion  forum  groups  on  learning,  both  actual  and  perceived,  course 
satisfaction,  and  sense  of  community  between  two  groups  participating  in  an  identical  online  professional 
development  Science  course.  Over  140  teachers  were  randomly  assigned  to  either  a discussion  forum  course  or  a 
non-discussion  forum  course  (control).  Between  the  two  groups,  results  from  this  study  show  a significant  difference 
in  the  sense  of  community  and  perceived  learning  of  water  conservation  strategies.  However,  the  results  also  find  no 
significant  difference  in  actual  learning,  course  satisfaction,  or  perceived  learning  of  Arizona  geography  and 
Arizona  water  systems.  This  paper  provides  a theoretical  background  of  the  use  of  discussion  forums  in  online 
courses,  elaborates  on  the  participant  pool  and  findings,  and  describes  implications  for  future  research. 

Introduction 

Professional  development  is  an  integral  part  of  a teacher’s  training  process.  However,  many  professional 
opportunities  fail  because  they  are  a “one-shot,  one-size  fits  all”  approach,  which  doesn't  allow  participants  the  time 
to  practice  or  implement  what  they  learn  (Lock,  2006).  The  professional  development  sessions  are  either  squeezed 
into  a short  time  window  before  school  or  directly  after  school,  when  teachers  have  already  completed  a full  day  of 
work.  Additionally,  with  lesson  planning,  conferences,  and  other  professional  duties,  time  is  limited  for  teachers  to 
explore  what  they  have  learned  after  they  attend  the  professional  development.  Taking  part  in  an  online  professional 
development  course  allows  the  teacher  to  participate  during  a time  that  is  convenient  for  them,  allowing  access  to 
the  course  materials  at  any  time  (Gonzalez  & Morales,  2001).  However,  low  completion  rates  of  online  courses  have 
led  many  organizations  to  add  a social  interaction  element  that,  if  done  correctly,  can  help  provide  a more  authentic 
experience  (Githens,  2007). 

There  have  been  numerous  studies  that  have  yielded  two  conclusions  about  online  learning  versus  face-to- 
face  learning:  1)  online  learning  can  be  just  as  effective  and  2)  the  lack  of  face-to-face  contact  does  not  negatively 
impact  learning  (Moore  & Kearsley,  2005).  But  is  there  a difference  in  learning  if  two  groups  complete  the  same 
online  course  in  which  one  group  participates  in  a discussion  forum  while  the  other  group  does  not?  The  goal  of  this 
study  was  to  answer  this  question  by  comparing  scores  on  a pre-  and  post-assessments  as  well  as  written 
assignments.  Additionally,  results  from  a course  evaluation  will  help  compare  the  learner's  feelings  of  his/her  sense 
of  community  in  his/her  course  and  to  examine  if  this  data  has  any  impact  on  the  assessment  results. 

Discussion  forums  and  learning 

Discussion  forums  are  a typical  component  of  online  courses  that  provide  participants  with  the  ability  to 
interact  socially  and  academically  with  other  distance  learners  in  the  course.  Picciano  (Piciano,  2002  as  stated  in 
Moore  & Kearsley,  2005)  studied  the  relationship  between  the  amount  of  social  interaction  and  the  overall 
performance  in  an  online  course.  He  concluded  that  students  who  participated  at  a higher  level  typically  scored 
better  on  written  assignments  but  that  there  were  no  differences  in  how  they  performed  on  objective  tests.  However, 
if  one  were  to  categorize  discussion  forums  as  a form  of  online  collaboration,  one  study  has  revealed  that 
collaborative  learning  does  not  always  lead  to  the  desired  outcomes  of  increased  learning  (Du,  Zhang,  Olinzock,  & 
Adams,  2008). 

Social  interaction  in  discussion  forums  is  hypothesized  to  provide  a foundation  for  learning,  meaning  that 
the  more  the  participants  feel  there  is  sense  of  social  interaction,  the  greater  the  effort  and  the  learning.  Additionally, 
the  critical  thinking  involved  in  discussion  forum  use  is  seen  as  directly  related  to  learning  (Wise,  Chang,  Duffy,  & 
Del  Valle,  2004).  Mitchem  (2008)  states: 

The  most  valued  aspects  of  online  discussion  groups  are  the  flexibility  and  the  convenience 

of  asynchronous  participation,  and  inclusion  of  diverse  participants  and  perspectives  from 

multiple  locations. 
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Githens  (2007)  offers  five  reasons  why  people  value  online  courses  that  include  discussion  forums.  First, 
participants  value  reading  experiences  from  other  participants.  Second,  participants  value  the  socialization  with 
others  in  the  field.  Third,  participants  often  provide  complex  and  contradictory  perspectives.  Fourth,  participants 
who  do  not  engage  in  discussion  forums  are  often  still  engaged  with  the  forums.  Fifth,  learners  feel  online 
discussion  forums  are  an  “intellectually  safe”  place  to  post  comments. 

While  social  interaction  may  lead  to  more  participation  in  discussion  forums,  the  research  is  not  clear  as  to 
the  impact  they  have  on  learning,  even  though  the  assumption  is  that  this  interaction  is  a critical  part  to  the  learning 
experience  (Dennen,  2008).  Caspi  and  Blau  (2008)  suggest  that  it  may  only  be  a “socioemotional”  source  of 
perceived  learning  and  that  cognitive  learning  is  unaffected. 

Method 


Participants 

This  study  included  142  K-12  teachers  (13  male,  129  female)  from  various  locations  throughout  a state  in 
the  southwestern  United  States.  The  teachers  were  recruited  through  an  advertisement  posted  within  IDEAL,  the 
state’s  educator  portal.  Any  teacher  who  was  interested  in  participating  in  the  course  and  research  study  was  exempt 
from  paying  the  $50  course  registration  fee.  Of  the  142  participants,  74  participated  in  the  intervention  group  and  68 
participants  were  in  the  control  group.  Although  the  course  was  designed  for  intermediate  level  (grades  4-6),  it  was 
available  to  all  teachers  listed  in  the  portal's  database  with  the  distribution  including  eight  primary  teachers,  58 
intermediate  teachers,  33  middle  school  teachers,  and  43  high  school  teachers.  The  course  was  offered  over  a four- 
week  period  in  the  summer  of  2009. 

Course  Design 

The  online  Science  course  was  developed  in  the  SAKAI  learning  management  system.  The  course  layout 
included  integrating  materials  taken  from  a pre-established  curriculum  used  in  face-to-face  trainings.  Digital  videos 
and  images  were  embedded  in  the  learning  management  system  to  supplement  the  curriculum.  Prior  to  the  start  of 
the  study,  a pilot  study  was  conducted  to  test  the  course  materials  and  data  instruments. 

Research  Design  and  Instrumentation 

This  quasi-experimental  study  used  comparative  methods  to  analyze  1)  learning  between  pre-  and  post- 
assessment scores,  2)  scores  on  a culminating  project,  and  3)  responses  to  a course  evaluation.  Each  of  the 
instruments  below  was  tested  in  a pilot  study  conducted  prior  to  the  start  of  this  study. 

First,  in  order  to  analyze  learning  between  groups,  the  participants  were  given  a 25-question  multiple- 
choice  pre-assessment  (00  of  the  course  content,  then  randomly  assigned  to  either  the  intervention  or  control  group. 
Each  group  received  identical  content  with  the  intervention  group  participating  in  discussion  forum  assignments  (X). 
At  the  conclusion  of  the  course,  each  participant  was  then  asked  to  complete  a post-assessment  (CF)  that  was 
identical  to  the  pre-assessment. 

Next,  each  participant  was  required  to  complete  a final  project  at  the  conclusion  of  the  course  (X).  The 
projects  were  scored  using  a rubric  by  two  course  facilitators.  The  scores  on  the  final  projects  were  compared 
between  the  intervention  and  control  groups  (0 1 ) . 

Finally,  each  participant  was  asked  to  submit  a course  evaluation  to  give  their  overall  satisfaction  with  the 
course,  their  perceived  learning,  and  their  perceived  sense  of  community  at  the  conclusion  of  the  course  (X).  The 
ratings  of  these  areas  were  compared  between  groups  (0i). 

Procedure 

The  teachers  registered  for  the  Arizona  Water  Story  online  course  through  IDEAL,  Arizona  educator 
portal.  Initially,  176  teachers  registered  and  started  the  course.  Due  to  the  high  number  of  participants  and 
enrollment  limitations,  six  total  sections  were  created:  three  intervention  sections  (discussion  forum)  and  three 
control  sections  (no  discussion  forum).  The  participants  were  randomly  assigned  to  one  of  the  six  sections  using  an 
online  random  generator. 
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Once  the  participants  were  assigned  a section,  each  individual  completed  a 25  multiple-choice  pre- 
assessment of  the  course  information  to  determine  a baseline  set  of  data.  At  the  conclusion  of  the  course,  an  identical 
post-assessment  would  be  administered  to  compare  with  the  pre-assessment  results  to  determine  if  learning  occurred 
during  the  course. 

Each  section  of  the  course  received  identical  information  and  materials  during  the  course.  Each  group  was 
also  expected  to  complete  a post-assessment  and  final  project  as  part  of  the  final  evaluation.  The  only  difference 
between  the  two  groups  was  that  intervention  group  participated  in  discussion  forum  activities. 

The  four-week  online  course  was  self-paced,  meaning  that  participants  could  work  at  their  own  pace  to 
complete  the  post-assessment  and  final  project  by  the  end  of  the  course,  and  divided  into  five  units:  1)  Introduction, 
2)  Arizona  Geography  and  Water  Sources,  3)  The  Hydrologic  Cycle,  4)  The  Arizona  Water  Story,  and  5)  Arizona 
Water  Use  Today.  However,  the  intervention  group  had  five  discussion  forum  assignments  that  were  due  at  the  end 
of  each  week.  Each  discussion  forum  assignment  required  the  participants  to  individually  respond  to  a prompt 
related  to  one  of  the  course  units.  Additionally,  the  participants  in  the  intervention  group  were  asked  to  reply  to  other 
group  members’  posts. 

At  the  conclusion  of  the  course,  all  participants  were  required  to  complete  the  post-assessment  and  final 
project  that  would  be  evaluated  and  used  to  assess  their  understanding  of  the  content  and  pedagogy.  The  participants 
were  then  asked  to  complete  and  evaluation  of  the  online  course. 

Results 


Full  Intervention  Effectiveness 

The  effectiveness  of  the  intervention  on  participants  completing  the  course  was  determined  by  comparing 
scores  on  pre-  and  post-assessment  scores  and  final  project  scores  for  both  the  intervention  and  control  groups. 

Learning  results  on  the  pre-  and  post-assessment.  A one-way  analysis  of  variance  (ANOVA)  was 
conducted  to  evaluate  the  differences  between  the  intervention  and  control  groups  on  the  pre-  and  post-assessment. 
The  independent  variable,  group  assignment,  included  two  levels:  1)  discussion  forum  and  2)  no  discussion  forum. 
The  dependent  variable  was  the  change  in  scores  between  the  pre-  and  post-assessment.  The  ANOVA  for  the  main 
effect,  group  assignment,  was  not  significant:  F(l,140)  = .031,  p=.861.  The  graph  (Figure  1)  shows  the  average 
change  from  pre-  and  post-assessment  for  both  groups. 

Learning  results  on  the  final  projects.  Next,  a one-way  analysis  of  variance  (ANOVA)  was  conducted  to 
evaluate  the  differences  between  the  intervention  and  control  groups  on  the  final  project.  The  independent  variable, 
group  assignment,  included  two  levels:  1)  discussion  forum  and  2)  no  discussion  forum.  The  dependent  variable  was 
the  score  on  the  final  project.  The  ANOVA  for  the  main  effect,  group  assignment,  was  again  not  significant: 
F(l,140)  = .629,  p=.428.  The  graph  (Figure  1)  shows  the  average  score  on  the  final  project  for  both  groups. 
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Figure  1.  Bar  graph  displaying  assessment  scores  by  group 


Satisfaction  and  Perceived  Learning 

All  of  the  participants  who  initially  registered  for  the  course  were  asked  to  complete  a course  evaluation  at 
the  course’s  conclusion.  Of  the  176  participants  who  registered,  146  of  those  submitted  the  evaluation  (72  from  the 
intervention  and  74  from  the  control  group).  The  survey  asked  participants  to  rate  their  satisfaction  with  the  course, 
their  perceived  learning,  and  their  perceived  sense  of  community  during  the  course  using  a 5-point  Likert-scale  (1: 
strongly  agree,  2:  agree,  3:  neither  agree  nor  disagree,  4:  disagree,  and  5:  strongly  disagree).  Example  questions 
include:  “I  learned  new  ways  to  conserve  water  from  the  Arizona  Water  Story  online  course”  and  “I  felt  a sense  of 
community  in  this  online  course”. 

Overall  course  satisfaction.  The  average  course  satisfaction  for  the  intervention  group  was  1.41  out  of  five 
while  the  control  group  average  was  1.38  out  of  five.  A one-way  analysis  of  variance  (ANOVA)  was  conducted  to 
evaluate  the  differences  between  the  intervention  and  control  groups  on  overall  satisfaction  of  the  course.  The 
independent  variable,  group  assignment,  included  two  levels:  1 ) discussion  forum  and  2)  no  discussion  forum.  The 
dependent  variable  was  average  rating  on  overall  satisfaction.  The  ANOVA  for  the  main  effect,  group  assignment, 
was  again  not  significant:  F(l,142)  = .041,  p=.839. 

Perceived  learning.  A one-way  analysis  of  variance  (ANOVA)  was  conducted  to  evaluate  the  differences 
between  the  intervention  and  control  groups  on  perceived  learning  of  the  course.  The  independent  variable,  group 
assignment,  included  two  levels:  1)  discussion  forum  and  2)  no  discussion  forum.  The  dependent  variable  was 
average  rating  on  three  different  areas  of  perceived  learning:  1)  Arizona  geography,  2)  The  Arizona  Water  System, 
and  3)  Water  conservation  strategies.  The  ANOVA  for  the  main  effect,  group  assignment,  was  not  significant  for 
either  learned  Arizona  geography  or  Arizona  water  systems:  F(  1,143)  = 1.253,  p=.265  and  F(  1,143)  = .001,  p=.980, 
respectively.  However,  the  ANOVA  for  the  main  effect,  group  assignment,  for  water  conservation  strategies,  was 
significant:  F(l,143)  = 4.224,  p=.042. 

Sense  of  community.  Lastly,  a one-way  analysis  of  variance  (ANOVA)  was  conducted  to  evaluate  the 
differences  between  the  intervention  and  control  groups  on  sense  of  community  in  the  course.  The  independent 
variable,  group  assignment,  included  two  levels:  1)  discussion  forum  and  2)  no  discussion  forum.  The  dependent 
variable  was  average  rating  a sense  of  community  question  taken  from  the  course  evaluation.  The  ANOVA  for  the 
main  effect,  group  assignment,  was  significant:  F(l,143)  = 29.950,  p<.001. 
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Table  1. 

Course  evaluation  scores 

Questions  DF  No  DF 

Overall,  I am  pleased  with  this  online  experience.  1.41  1.38 

I learned  about  Arizona  geography  from  this  online  course.  1.34  1.44 

I learned  about  the  Arizona  water  system  from  this  online  course.  1 .28  1 .28 

I learned  new  ways  to  conserve  water  from  this  online  course.  1.49  1.76 

I felt  a sense  of  community  in  this  online  course.  2.24  3.19 

All  scores  are  out  of  5 with  1 =strongly  agree  and  5=strongly  disagree 


Dropouts.  The  overall  completion  rate  for  the  online  course  was  82%  (142  of  174).  The  group  that 
participated  in  discussion  forums  had  a completion  rate  of  86%  (74  of  86)  while  the  group  that  did  not  participate  in 
discussion  forums  had  a completion  rate  of  77%  (68  of  88).  An  anonymous  survey  was  distributed  to  these 
participants  so  that  they  could  provide  feedback  as  to  why  they  were  unable  to  complete  the  course.  Of  the  32 
dropouts,  14  responded  to  the  survey.  The  largest  contributor  to  not  finishing  the  course  was  previous  time 
commitments,  as  shown  in  Table  2. 


Table  2. 

Reasons  for  not  completing  the  course 

Reason  Number 

Previous  time  commitments.  9 

Content  was  not  appropriate  to  my  grade  level.  0 

Content  was  not  interesting  to  me.  0 

The  course  was  too  difficult  to  navigate.  1 

The  course  objectives  were  unclear.  0 

The  course  assignments  were  not  relevant  to  my  teaching  position.  2 

I wanted  to  participate  in  discussion  fomms  but  was  not  assigned  to  that  group.  2 

I did  not  want  to  participate  in  discussion  forums  but  was  assigned  to  that  0 

group. 

I did  not  receive  help  when  I asked  for  it.  0 

Would  you  take  the  course  if  it  were  offered  again?  1 1 

N=14  dropouts  responded  to  the  survey 


Discussion 

The  generalization  that  can  be  discerned  from  the  pre-  and  post-assessment  data  collected  in  this  study 
indicates  that  participating  in  an  online  discussion  forum  does  not  lead  to  higher  gain  in  achievement  scores  between 
a pre-  and  post-assessment.  Complimentary  to  this  information  is  lack  of  difference  in  perceived  learning  between 
the  two  groups  on  Arizona  geography  and  the  Arizona  water  system,  as  shown  in  the  course  evaluation.  Flowever, 
there  was  a significant  (p<.05)  difference  in  perceived  learning  between  the  two  groups  when  the  participants  were 
asked  about  water  conservation  strategies.  One  reason  that  the  data  shows  no  difference  for  a majority  of  the  results 
is  because  all  participants  voluntarily  enrolled  in  the  online  course.  Lim  and  Kim’s  (2003)  study  shows  that  as 
motivation  increases  perceived  learning  also  increases.  One  can  then  infer  that  all  participants  were  inspired  to  study 
the  information  and  therefore  all  participants  were  especially  diligent  in  learning  the  information.  Additionally,  Lim 
and  Kim's  study  concluded  that  female  participants  show  a significantly  higher  degree  of  learning  than  male 
participants  when  motivation  levels  increase.  This  is  important  for  the  current  study  because  of  the  larger  percentage 
of  female  participants  in  the  study  (91%). 

The  results  on  the  final  project  scores  that  evaluated  pedagogical  knowledge  of  teaching  the  content  were 
also  not  affected  by  placement  in  either  group.  The  discussion  forum  prompts  were  designed  to  allow  participants  to 
post  ideas  on  how  they  would  teach  the  main  course  objectives  in  their  classrooms.  One  would  assume  that 
discussing  pedagogical  strategies  with  other  classmates  would  help  the  participants  learn  new  strategies  for  teaching 
the  main  course  objectives.  However,  the  results  of  the  study  do  not  substantiate  this  claim.  Dennen  (2008)  mentions 
that  while  discussions  are  an  important  part  of  the  online  experience,  the  “act  of  writing”  is  not  the  only  factor  that 
contributes  to  learning. 
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Both  groups  responded  that  overall,  they  were  pleased  with  the  course.  However,  the  group  that 
participated  in  discussion  forums  reported  a significantly  higher  sense  of  community  (p<.001)  than  the  group  who 
did  not  participate  in  the  discussion  forum.  An  inference  that  can  be  made  from  this  is  that  the  participants  in  the 
discussion  forum  group  were  able  to  share  thoughts  and  ideas  with  one  another  similar  to  the  way  participants  would 
in  a traditional  setting.  Dennen  (2008)  concurs  that  while  discussions  are  considered  a necessary  component  to 
online  courses,  they  are  typically  added  for  no  other  reason  that  to  provide  a connection  between  participants. 

It  is  also  important  to  note  that  significantly  more  participants  from  the  control  group  dropped  out  than  did 
in  the  intervention  group,  20  and  12  respectively.  According  to  the  survey,  the  major  reason  for  participants  not 
completing  the  course  was  a lack  of  time.  However,  since  only  14  of  the  32  dropouts  responded  to  the  survey,  it  is 
difficult  to  determine  if  this  is  the  main  reason  for  not  completing  the  course. 

Further  research  on  online  discussion  forums  and  learning  is  important  to  determine  if  the  data  collected  in 
this  study  can  be  generalized  across  other  populations.  Additionally,  the  there  is  a need  for  more  rigorous  and 
objective  standards  in  terms  of  grading  discussion  forums.  Finally,  although  the  course  facilitator  was  not  an  integral 
part  of  this  study,  studies  need  to  be  conducted  to  determine  if  an  instructor’s  role  in  the  discussion  forum  increases 
learning,  and  if  so,  how  that  role  should  be  clearly  defined. 
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Abstract 

This  article  will  document  the  design  of  a distance  education  Out  of  School  Suspension  Program  and  will 
map  out  the  purpose  of  learning  organizations  (systems  approach).  It  discusses  primary  objectives,  inputs,  processes 
and  outputs  of  the  systems.  In  addition,  the  article  focuses  on  supported  learning  theories  and  explains  how  mega, 
macro,  and  micro  communities  are  integrated  into  the  system. 

In  addition,  it  describes  the  system  in  its  entirety,  including  elements  and  relationships,  interdependency, 
vision  and  learning  paths.  Following  the  description  of  the  system  is  the  mega,  macro  and  micro  communities’ 
makeup. 

Finally,  an  assessment  is  presented  of  the  system’s  particular  blueprint  and  efficacy,  including  the  impact  of 
special  education  on  the  school  and  society. 


Introduction 

Anyone  visiting  our  schools  of  today  may  notice  that  frequently,  special  education  students  demonstrated 
continuous  disruptive  behaviors.  As  well  as  causing  continuous  disruptive  behavior,  visitors  may  notice  acts  of 
violence,  such  as  a student  throwing  a chair  at  another  student.  Under  both  the  No  Child  Left  Behind  Act  and  the 
Individuals  with  Disabilities  Education  Act  and 

According  to  federal  and  state  regulations,  the  removal  of  a student  for  more  than  ten  (10)  days 
because  of  a violation  of  a code  of  student  conduct  is  regarded  as  a change  in  educational 
placement.  Because  of  this  change  in  placement,  the  action  of  removal  triggers  the  need  for  an  IEP 
meeting  where  the  IEP  team  must  determine  whether  the  behavior(s)  that  led  to  the  suspension  is 
related  to  or  is  because  of  the  student’s  disability.  Additionally,  the  team  must  meet  to  consider 
whether  the  conduct  in  question  was  because  of  some  type  of  failure  by  the  LEA  to  implement  the 
IEP  of  the  student.  If  so,  then  the  IEP  team  must  either  1)  conduct  a FBA  (functional  behavioral 
assessment)  or  implement  a BIP  (behavioral  intervention  plan)  for  the  child;  or  2)  if  a BIP  is 
already  in  place,  review  and  revise  it  to  address  the  behavior  AND  reinstate  the  child  to  the 
educational  placement  from  which  the  child  was  removed.  (34  C.F.R.  §300. 530(e))  (Gibson,  2007, 

N.P.  f 7) 

This  article  will  introduce  a distance  education  system,  titled  Virtual  Out  of  School  Suspension  Program, 
designed  to  deliver  a professional  learning  environment  within  a K-12  public  school  system.  Distance  education  is 
on  the  rise  and  rapidly  developing  approaches  to  deliver  instruction  across  the  world.  (Smaldino,  Lowther  & Russell, 
2008).  The  system  is  designed  to  afford  the  school's  principal  the  opportunity  to  send  students  home  for  a period 
longer  than  ten  days  within  a flexible  learning  environment,  as  an  extension  of  a low  socio-economic  elementary 
school. 


Methodology 


Setting 


The  writer  is  a student  at  a large  non-profit  university  in  the  Southeastern  United  States  with  a nationally 
recognized  program  providing  leadership  and  scholarship  in  the  systematic  application  of  instructional  technology  to 
the  research-based,  best-practices  delivery  of  instruction  in  virtual  schools,  higher  education  distance  education,  and 
e-learning  in  business,  industry,  and  government  (Nova,  2009). 
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One  of  the  mandated  courses  for  doctoral  students  with  a concentration  in  instructional  technology  distance 
education  is  a 3-credit  course  in  system  analysis  and  design.  This  course  provided  the  writer  with  skills  for  the 
analysis  and  design  of  an  educational  and  instructional  system.  The  key  focus  was  on  critical  elements  in  the 
structure  of  a distance  education  delivery  system;  analysis  of  educational  and  instructional  systems;  relationships  of 
subsystems  within  an  educational  system;  design  of  an  educational  system;  design  of  instructional  systems;  and 
evaluation  and  continuous  improvement  of  a system. 

The  writer  analyzed  a low  socio  economic  status  elementary  school  located  in  Southeastern  United  States, 
identified  steps  in  designing  a system,  created  a valid  distance  education  system  and  identified  subsystems  that  may 
impact  the  student's  performance  within  the  system.  Since  the  writer  worked  closely  with  special  education 
students'  she  chose  to  develop  a virtual  out  of  school  suspension  program. 

Purpose 

The  purpose  of  this  article  is  to  introduce  to  the  reader  a distance  education  system,  titled  Virtual  Out  of 
School  Suspension  Program  (V.O.S.S.P.)  that  will  enable  students  to  continue  their  coursework  while  not  physically 
in  attendance  at  their  current  school. 

The  writer  believes  that  requirements  in  the  current  No  Child  Left  Behind  (NCLB)  law  may  have  affected 
student's  safety.  Currently,  NCLB  asks  the  individual  states  and  schools  to  develop  standards  for  safety,  apply 
research-based  programs,  gather  and  evaluate  data,  and  improve  on  safety  problems  identified  by  schools  and  states 
in  order  to  make  “Adequate  Yearly  Progress”  (U.S.  Department  of  Education  Website,  2002,  pg  31). 

According  to  the  U.S.  Department  of  Education  Website’s  the  U.S.  Department  of  Education  is  required  to 
conduct  an  evaluation  on  the  impact  of  schools  funded  under  Safe  and  Drug  Free  Schools  (SDFS).  It  states  that 
programs  must  demonstrate  a reduction  in  school  violence.  Both  the  NCLB  and  the  Individuals  with  Disabilities 
Education  Act  (IDEA)  (2004)  question  the  long  existing  practice  of  school  suspensions.  Under  NCLB,  it  is  difficult 
to  place  special  education  students  in  out  of  school  suspension  for  any  time  greater  than  ten  days  per  school  year. 
Furthermore,  a concern  is  that  out  of  school  suspension  does  not  allow  students  to  continue  with  their  necessary 
coursework.  Thus,  the  writer  has  designed  a distance  education.  Out  of  School  Suspension  Program  which  enables 
students  the  opportunity  to  continue  their  education  while  fulfilling  the  requirements  of  their  suspension  and 
allowing  school  administrators’  the  chance  to  assign  out  of  school  suspensions  for  a period  longer  than  the 
maximum  ten  days.  In  the  near  future,  an  actual  operation  within  our  schools  would  lead  to  a calmer,  safer,  and 
more  productive  school  climate. 

The  Development  of  the  Virtual  Out  of  School  Suspension  Program 

Resources 

The  budget  of  the  V.O.S.S.P.,  supported  through  federal,  state,  and  local,  funds,  based  on  a forecasted  plan 
for  the  program  and  created  each  fiscal  year.  The  budget  is  based  on  predicted  student  enrollment.  The  budget  will 
incorporate  faculty  trainings,  salaries,  distance  education  materials,  a local  area  network  (LAN),  laptops,  and  other 
miscellaneous  technological  items. 

Community 

The  V.  O.  S.  S.  P's  office  is  housed  in  a low  socio-economic  urban  community  comprised  of  government 
subsidized  housing  and  private  homes  located  in  the  Southeastern  United  States. 

History 

According  to  Gibson  (2007),  “A  limited  amount  of  recourse  is  available  to  the  student  through  federal, 
state  and  district  regulations”  (Gibson,  2007).  Thus,  after  a special  education  student  is  removed  for  more  than  ten 
days  from  school  they  are  regarded  as  a change  of  placement.  This  action  triggers  the  need  for  an  Individualized 
Education  Plan  (IEP)  meeting  where  the  IEP  team  determines  whether  the  behaviors  that  led  to  the  suspension  was  a 
result  of  the  student's  disability.  Then,  the  team  decides  if  this  behavior  was  due  to  improper  implementation  of  the 
IEP.  If  so,  the  IEP  team  must  conduct  a functional  behavioral  assessment  and  implement  a behavior  intervention 
plan  (BIP).  If  the  student  already  has  a behavior  intervention  plan,  the  need  is  to  review  and  revise  it  and  reinstate 
the  child  to  their  home  school. 

Unfortunately,  the  schools  climate  survey  report  for  the  2007-08  administration  form  at  the  district  level 
states  35%  of  the  teachers  at  the  low  socio  economic  elementary  school  felt  that  adequate  disciplinary  measures  are 
not  being  used  to  deal  with  disruptive  behavior  (district,  2008,  P.  4,  (|[  1). 
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So,  the  questions  are,  would  a virtual  out  of  school  suspension  program  afford  the  principal  a viable 
alternative  to  having  students  involved  in  disruptive  behaviors  at  their  school?  Would  V.  O.  S.  S.  P.  enable  students 
to  continue  their  education  and  learn  conflict  resolution  skills  while  in  suspension?  Would  a program  such  as  this  be 
valuable  to  schools  across  America? 


Description  of  the  System 


Elements  and  Relationships 

The  V.O.S.S.P.  design  is  made  up  of  administrative  leadership,  technical,  and  curriculum  support  teams. 
The  administrative  team  will  decide  which  students  will  attend  the  V.O.S.S.P  and  for  how  long.  The  leadership  team 
will  provide  programs  that  empower  teachers  and  students  with  technology.  Professional  development  and  distance 
learning  opportunities  highlight  the  V.  O.  S.  S.  P.  and  will  set  the  stage  for  teachers,  students,  and  administrators  to 
create,  experience,  and  advance  21st  century  learning.  The  technical  support  team  provides  students,  teachers, 
administrators  and  parents  with  the  tools  they  need  to  integrate  technology  into  their  daily  work  through  reliable 
technology  resources,  such  as  laptops,  internet  connections.  The  learning  and  teaching  aspect  will  include 
information  skills  standards  and  curriculum,  instructional  models,  program  integration  with  flexible  scheduling, 
assessment  rubrics,  information  skills  lesson  plans,  subject  area  lesson  plans,  technology  that  will  benefit  all 
stakeholders.  The  curriculum  support  and  assessment  team  will  work  to  ensure  appropriate  online  curriculum  is 
accessible  and  used  to  assure  the  students'  goals  and  benchmarks  are  met. 

There  will  be  a continuous  flow  of  equal  interdependence  between  the  inputs  and  outputs  of  the 
communities  (mega,  macro,  micro).  The  V.O.S.S.P  will  operate  under  the  district  systems  policies  and  procedures. 
The  operating  budget  of  the  virtual  out  of  school  suspension  program  will  be  dependent  on  the  district  and  the 
Elementary  school's  funding  as  well  as  funding  obtained  from  mini  grants  written.  Interactions  between  the 
Elementary  and  the  V.O.S.S.P  will  be  ongoing  and  expected  to  occur  at  least  weekly.  The  decisive  output  of  the 
Macro  Community  is  the  proposed  V.  O.  S.  S.  P.  The  Micro  Communities  outputs  (resources)  flow  into  the  Macro 
Community.  Inputs  from  the  Mega  Community  (school  district)  will  be  outputs  for  the  Macro  Community.  All 
levels  are  interdependent  and  evidenced  through  open  boundaries.  Communications  and  resources  will  allow  the 
system  to  function  efficiently. 

Vision 

The  vision  of  the  V.  O.  O.  S.  P.  is  to  educate  special  education  students  who  disrupt  the  traditional  school 
classroom.  It  provides  a focused  learning  environment  and  a thorough  education  that  meets  The  student's 
educational  state  standards. 

Learning  Paths 

The  V.O.S.S.P.  is  designed  to  improve  a student's  education  through  online  instruction  while  incorporating 
diversified  instruction,  individual  learning  styles  and  personalized  instruction.  The  learning  paths  are  synchronous, 
occurring  simultaneously  with  A.  B.  Elementary'  students  learning  paths.  The  curriculum  and  materials,  supplied  by 
the  school  district  and  carried  out  in  an  “Elluminate”  online  setting.  The  Elluminate  live  software  enables  instructors 
and  students  to  interact  and  collaborate  in  real  time  to  add  synchronous  content  to  asynchronous  distance  learning  or 
combine  blended  online/onsite  learning  activities.  The  designers  of  Elluminate  believe  it  will  engage  teachers  to 
promote  active  learning  and  improve  student  performance. 

The  curriculum  support  team  will  refer  to  the  state  and  district  instructional  pacing  guide,  broken  down  into 
nine-week  segments.  The  curriculum  support  team,  home  school  and  the  district  will  devise  a calendar,  which 
contains  weekly  or  bi-weekly  benchmarks,  to  assess  student  mastery  and  determine  which  skills  and  content  need  to 
be  reassessed. 

The  V.  O.  S.  S.  P.  will  use  a pacing  plan  and  the  student's  most  recent  school  work  to  drive  their 
instruction.  Additional  curriculum  in  the  area  of  conflict  resolution  will  be  added  to  the  student's  current  curriculum 
and  must  be  mastered  before  the  student  returns  to  their  home  school.  Faculty  development,  instructional  design,  and 
student  success  rates  will  maintain  ongoing  data  to  document  success  and/or  areas  in  need  of  improvement.  Ongoing 
professional  development  trainings  and  revisions  will  occur  throughout  the  school  year.  Student's  assessments  will 
be  ongoing  and  computer  generated  scores  will  be  entered  in  a database  each  Friday.  Student's  ability  and 
completion  of  assigned  (weekly)  curriculum  will  be  determined  through  assessments.  Generated  synchronous  with 
the  home  school  systems  grading  periods  is  a nine-  week  progress  report. 
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Supported  Learning  Theory 

The  three  theories  most  frequently  encompassed  are  Bruner's  constructionist  Skinner's  behavior 
conditioning  theories,  Piaget  and  Vgotsky.  The  constructivist  theory  is  used  foremost  in  supporting  the  students’ 
learning.  Bruner's  constructivist  theory  is  an  overall  framework  for  the  V.  O.  S.  S.  P.  's  instruction  because  it  is 
based  upon  the  study  of  cognition.  Much  of  Bruner's  theory  is  linked  to  child  development  research  (including 
Piaget  & Vgotsky).  This  new  distance  education  program  will  utilize  Bruner's  learning  theories  to  individualize 
student's  instruction  and  assure  success  and  ease  of  success  in  returning  to  A.  B.  Elementary. 

Another  learning  theory  applied  in  the  proposed  V.  O.  S.  S.  P.  is  often  observed  in  classrooms  for  students 
with  emotional/behavioral  disabilities.  This  learning  theory,  known  as  behavioral  or  operant  conditioning  (learning) 
occurs  when  a response  to  a stimulus  is  reinforced  (Skinner,  1938).  Essentially,  operant  conditioning  is  a simple 
reaction  system:  If  a reward  or  reinforcement  trails  the  response  to  an  incentive,  then  the  response  becomes  more 
likely  in  the  future.  The  students  will  be  awarded  daily  points  and  receive  a “prize”  is  received  after  a total  of  50 
points  is  achieved.  Prizes  will  consist  of  one  choice  between  a shortened  assignment  (student  doing  even  number 
problems  only),  throw  away  one  low  grade,  or  a coupon  requesting  parents/guardian  take  student  to  lunch  (value  of 
$5.00  or  less). 

Communities 

“While  no  level  of  the  IEM  (Mega,  Macro,  Micro,  Process,  or  Input)  is  more  important  than  the  others,  it  is 
critical  that  we  begin  with  the  Mega  level  when  we  plan  for  educational  success  by  defining  outcomes  as  the  results 
at  that  level”  (Kaufman,  et  al,  2001,  p.  59).  Successful  planning  within  the  Mega  community  (state,  district,  and 
communities)  which  is  considered  an  “open  system”  will  ensure  a focus  on  future-oriented  initiatives.  First,  there  is 
the  immediate  community  consisting  of  students  in  grades  K-5,  parents,  faculty,  administrators,  and  support  staff. 
Next,  are  MDCPS  and  federal  guidelines  concerning  school  suspensions?  Last,  is  The  Florida  Department  of 
Education  and  the  United  States  Department  of  Education  who  have  the  ability  to  revise  IDEA  and  oversee  school's 
compliance  with  IDEA'S  procedures. 

The  mega  level  (Figure  1)  consists  of  the  state,  district,  society  (communities)  and  policies/procedures 
involved  in  the  development  of  the  V.  O.S.S.P.  This  system  shares  missions  and  visions  with  the  Macro  and  Micro 
communities.  It  consists  of  inputs  and  outputs  that  nourish  each  other.  The  output  of  the  mega  level  will  feed  as  an 
input  to  the  macro  level  (V.  O.  S.  S.  P.). 

Kaufman,  et  al  (2001)  explain,  “if  we  begin  at  the  Mega  level  and  identify  the  outcomes  we  are  committed 
to  achieving,  then  we  can  “roll-down'  from  Mega  to  define  the  data  rather  than  relying  on  conjecture”  (p.59,  '12). 
They  also  explain  how  stakeholders  (internal  and  external)  of  an  educational  institution  want  students  to  be 
successful  in  school  and  society.  Thus,  the  discernable  outcome  within  the  Mega  systems  community  will  be  to  have 
each  student  believe  in  his/her  ability  to  achieve  and  become  a future  leader. 

At  the  Mega  level,  the  inputs  consist  of  revenue  awarded  from  the  state,  U.S.  Department  of  Education,  and 
private  grants.  Inputs  also  include  the  laws,  regulations  and  policies  set  by  these  agencies  and  societal  needs. 

Another  input  would  be  community  partners  and  their  assistance  with  coursework  planning. 

Processes  will  include  the  systems  needs,  policy  implementation,  procedures,  and  generating  jobs. 

Last,  come  the  outputs  that  will  lead  into  the  Macro  Community  inputs.  The  outputs  consist  of  the  budget, 
education,  employment  and  necessary  technologies  to  meet  the  state's  requirements. 

The  boundaries  would  be  the  open  lines  of  communication  between  all  the  communities  (Mega,  Macro,  and 
Micro).  Emails,  and  phone  conversations  will  be  ongoing  and  Elluminate  sessions  will  enhance  the  open  line  of 
communications.  Elluminate's  voice  system  will  assure  instructors  and  administrators  that  it  is  the  actual  student 
participating  in  the  assignments. 

According  to  Kauffman,  et  al  (2001),  “Effective  Macro  planning  flows  from  Mega  planning  and  is  derived 
from  the  results  it  defines”(p.91).  The  Macro  level  represents  the  V.  O.  S.  S.  P.,  which  flows  from  the  Mega 
community.  Lauritsen  (2001),  Sampson,  Raudenbush,  & Earls  (1997)  agree,  communities  are  complex  systems  that 
can  be  explained  in  numerous  ways.  Thus,  the  largest  structure  solely  devoted  to  the  Macro  community  is  the  V.  O. 
S.  S.  P.  and  all  of  its  components. 

The  V.  O.  S.  S.  P.  is  comprised  of  one  classroom,  within  a low  socio  economic  elementary  school.  The 
virtual  learning  is  web  based  using  a LAN  network.  Website,  Elluminate,  online  resources,  and  emails.  Students 
participating  in  the  V.  O.  S.  S.  P.  are  issued  a lap  top  computer  for  use  while  participating  in  the  program.  The 
laptop  will  be  designed  to  have  limited  accessibility  to  the  internet  which  directs  the  students  to  the  V.  O.  S.  S.  P. 
Students  who  choose  not  to  check  out  a laptop  due  to  liability  or  other  issues  may  access  the  V.  O.  S.  S.  P through 
their  local  library's  computer  systems  or  their  own  personal  home  computers/  laptops.  The  student's  special 
education  needs  such  as  Speech,  Language,  Physical  Therapy  and/or  Occupational  Therapy  will  be  accessible  to  the 
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student  via  their  home  school.  However,  it  will  be  the  parent's  responsibility  to  provide  transportation,  possibly, 
requiring  an  interim  IEP  to  clearly  state  the  change  in  transportation. 

The  V.  O.  S.S.  P.  department's  faculty  (instructional  and  non-instructional)  are  under  the  supervision  of  the 
three  program  support  (curriculum,  technical  and  leadership)  teams  who  are  guided  by  the  school’s  assistant 
principal.  The  assistant  principal  is  under  the  authority  of  the  Principal. 

There  is  a constant  flow  of  communication  between  the  communities.  The  Mega  Community  will  interpret 
federal  laws  and  establish  the  procedures  for  the  Micro  Community  to  operate  within  compliance  issues.  The  Micro 
Community  will  be  responsible  for  implementing  these  policies  and  procedures  which  in  turn  will  assure  the 
student's  IEP  and  benchmarks  are  being  met. 

The  Macro  level's  inputs  would  pertain  to  curriculum,  technical  and  leadership  team,  faculty  who  will  be 
teaching  and  the  systems  policies/procedures.  Processes  would  be  the  purchasing  and  implementation  of  curriculum, 
reviewing  goals,  objectives,  assessments,  and  training  development  necessary  based  on  staff,  parent,  and  student 
inputs.  Outputs  include  resources  needed,  modification  of  programs  and  staff  development  and  workshops. 

The  boundaries  would  be  the  open  lines  of  communication  between  all  the  communities  through  emails, 
phone  conversations,  and  Elluminate  sessions.  This  will  enhance  the  open  line  of  communications. 

Consider  next,  the  micro  communities  whose  direct  stakeholders  would  be  the  administrators,  parents, 
teachers,  community  groups,  faculty,  and  students.  The  indirect  stakeholders  consist  of  the  Parent  Teacher 
Association  (PTA),  Excellence  School  Advisory  Committee  (EES  AC),  and  community  leaders.  The  role  of  the 
stakeholders  (indirect  and  direct)  is  that  of  active  participants  and  reviewers. 

The  school  is  under  the  supervision  of  the  principal  and  assistant  principal  which  support  the  leadership 
team.  The  leadership  team  gives  input  and  helps  make  decisions  concerning  recommendations  for  school 
improvement.  The  three  support  persons  (curriculum,  technical,  and  leadership)  for  V.  O.  S.  S.  P.  are  responsible  for 
program  functions.  The  Micro  level  is  the  “workers”  (teachers),  for  example,  producing  lesson  plans  that  equal  a 
product. 

Communications  flow  between  the  communities.  The  micro  level  inputs  would  consist  of  instructional 
technologies,  course  policies,  and  procedures.  The  process  would  consist  of  planning,  evaluating  needs,  and  learning 
styles.  This  process  also  encourages  learning  and  provides  resources  and  the  student  transfer  of  knowledge.  The 
outputs  would  be  to  have  student  gains,  meet  the  NCLB  and  the  return  of  students  to  A.  B.  Elementary. 

Evidenced  through  daily  emails  is  the  open  communications  between  communities,  specifically  the  Micro, 
Macro  Communities,  and  weekly  staffing. 

Assessment  Systems 

Since  the  V.  O.  S.  S.  P.  is  considered  a temporary  placement  assessment  will  be  designed  similar  but  on  a 
smaller  scale  as  A.  B.  Elementary  and  its  traditional  assessments.  Student  participation  in  the  Florida 
Comprehensive  Assessment  Test,  biweekly  assessments,  student,  teacher,  and  parent  surveys  assures  quality  for  the 
macro  community  is  being  achieved. 

Quantitative  and  qualitative  data,  used  to  corroborate  the  V.  O.  S.  S.  P.  Administrators  and  the  three 
support  positions,  develop  an  overall  program  evaluation  for  the  instructors  to  implement.  Thus,  ensuring  that 
reliable  progress  is  made  throughout  the  students  time  spent  in  the  V.  O.  S.  S.  P.  Administrators  review  all  surveys 
and  make  suggestions  for  program  and/or  teacher  improvements.  The  assessments  will  demonstrate  the  need  for  the 
V.  O.  S.S.P. 
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The  Virtual  Out  of  School  Suspension  Program 

View  of  Macro  Community  in  Relation  to  Mega  and  Micro  Communities 
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Conclusion 

The  student's  continuous  disruptive  behaviors  in  their  school  seems  to  be  a problem  prevalent  today.  The 
evident  frustration  of  the  principal  while  required  to  follow  the  No  Child  Left  Behind  Act  demonstrates  the  need  to 
create  an  alternative  to  traditional  Out  of  School  Suspension  programs.  The  Virtual  Out  of  School  Suspension 
Program  was  developed  to  fill  such  a need. 

The  virtual  out  of  school  suspension  program,  based  on  a systematic  process,  constructs  knowledge  through 
building  on  what  they  know  in  an  attempt  to  improve  instruction.  It  focuses  on  the  student  and  uses  theories  (Bruner, 
Piaget  & Vgotsky)  to  determine  which  is  appropriate  for  learning,  and  uses  that  framework  to  drive  their 
instructional  decisions.  The  Mega,  Macro  and  Micro  communities  are  interdependent  with  each  others’  systems 
inputs  leading  to  another's  outputs  and  vice  versa.  They  conduct  ongoing  assessments.  Surveys  from  students, 
teachers,  and  parents  highlight  necessary  changes  in  the  system.  While  attending  the  V.  O.  S.  S.  P,  the  disruptive 
student  receives  conflict  resolution  skills,  participates  in  a professional  learning  environment,  and  returns  to  school 
with  no  gaps  in  education. 

In  conclusion,  working  as  a team,  the  elementary  school,  special  education  department  and  the  virtual  out 
of  school  suspension  program  create  a level  of  success  for  the  stakeholders;  students,  parents,  guardians,  faculty, 
administration  and  the  community. 
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Abstract 

This  descriptive  study  seeks  to  understand  the  differing  requirements  of  doctoral  programs  in  educational 
technology  related  fields  in  the  United  States.  Using  a document  analysis  approach  with  five  data  sources,  we  found 
59  different  institutions  (55  campus-based  and  4 online)  offered  doctoral  degrees  with  33  different  degree  titles  that 
satisfied  a foundational  set  of  curriculum  criteria.  Among  these  59  institutions,  total  credit  hours  requirements 
ranged  from  42  credit  hours  to  1 13  credit  hours  and  total  dissertation  credit  hours  spanned  from  1 to  45  credit  hours 
to  complete  a doctoral  degree.  Among  the  nine  universities  who  offered  both  Ph.D.  and  Ed.D.  degrees,  the 
requirements  of  the  total  credit  hours  were  varied,  but  the  same  or  higher  total  dissertation  hours  were  required  for 
the  Ph.D.  program  than  for  the  Ed.D.  program. 


Introduction 

Potential  graduate  students  face  a plethora  of  challenges  when  trying  to  find  and  decide  upon  a doctoral 
studies  program  in  educational  technology.  What  exactly  is  educational  technology?  According  to  the  Definition  and 
Terminology  Committee  of  the  Association  for  Educational  Communications  and  Technology  (2007),  “Educational 
technology  is  the  study  and  ethical  practice  of  facilitating  learning  and  improving  performance  by  creating,  using, 
and  managing  appropriate  technological  processes  and  resources”  (p.  1).  However,  the  definition  of  the  field  of 
practice  continues  to  evolve  and  change  in  tandem  with  changes  in  education  and  technology.  Reiser  (2007)  believes 
that  it  is  unlikely  that  professionals  will  ever  reach  a consensus  through  thoughts  or  actions  on  a single  definition. 

Coupled  with  the  elusive  definition  of  the  field,  prospective  doctoral  students  and  potential  employers  often 
are  confounded  by  the  myriad  of  naming  conventions  for  higher  educational  programs  in  our  field.  When  searching 
for  doctoral  programs,  students  typically  find  the  names  of  educational  technology,  learning  sciences,  instructional 
technology,  curriculum  and  instruction,  or  instructional  design  and  technology  (IDT).  According  to  Persichitte 
(2007),  these  titles  represent  responses  to  the  changes  in  the  field,  and  it  is  perhaps  impossible  to  separate  the 
application  of  the  field  from  the  systemic  changes  that  are  intrinsic  to  it.  Nonetheless,  individuals  seeking  doctoral 
degrees  in  the  field  of  educational  technology  face  challenges. 

One  of  the  challenges  is  that  students  must  try  to  wade  through  differing  program  and  degree  titles  to  find  the 
curriculum  within  an  educational  technology  doctoral  degree  program.  They  evaluate  the  core  requirements  as  well 
as  investigate  the  extent  of  the  research  curriculum  and  the  intensity  of  the  dissertation.  One  way  to  consider  a 
meaningful  educational  technology  curriculum  is  to  place  it  in  a professional  competency-based  framework,  such  as 
those  offered  by  the  International  Board  of  Standards  for  Training,  Performance  and  Instruction  (IBSPTI), 
International  Society  for  Performance  Improvement  (ISPI),  the  American  Society  for  Training  and  Development 
(ASTD),  and  the  Association  for  Educational  Communications  and  Technology  (AECT).  Davidson-Shiver  and 
Rasmussen  (2007)  suggest. 

Competencies  for  IDT  can  be  used  as  follows:  ...To  provide  academic  programs  with  information  to  develop 
and  evaluate  curriculum,  courses,  internships,  and  program  requirements.  To  provide  academic  programs 
with  a basis  for  program  evaluation  and  planning.  . . .To  provide  IDT  academicians  with  a direction  for 
forming  a research  agenda  (p.  279). 

However,  they  also  pointed  out  that  the  lack  of  common  titles  for  programs  at  colleges  and  universities  make 
it  difficult  for  potential  students  to  match  their  goals  with  a specific  professional  competency  framework.  Caffarella 
(1999)  reported  that  universities  offered  doctoral  degrees  with  a variety  of  titles  encompassing  the  field  of 
educational  technology.  The  titles  of  degrees  vary  from  educational  technology  to  instructional  design  to 
information  and  learning  sciences,  and  there  is  considerable  debate  over  which  terms  are  more  appropriate  to  use  in 
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identifying  the  field  (Carr-Chellman,  2006).  For  example,  in  a debate  with  Merrill  about  the  future  of  the  field, 
Wilson  argued  that  the  growth  of  graduate  programs  in  the  field  is  in  both  instructional  design  and  technology,  and 
in  learning  sciences  (Merrill  & Wilson,  2007). 

Another  challenge  faced  by  those  seeking  advanced  degrees  in  the  field  is  deciding  which  type  of  degree  to 
pursue;  the  Ph.D.  or  the  Ed.D.  and  understanding  the  differences.  Some  institutions  offer  both  doctoral  degrees 
(Ph.D.  - Doctor  of  Philosophy  and  Ed.D.  - Doctor  of  Education)  while  most  offer  one  or  the  other.  In  their  Carnegie 
Initiative  on  the  Doctorate  study,  Shulman,  Golde,  Bueschel,  and  Grabedian  (2006)  explained  that  the  goal  of  the 
Ed.D.  is  to  train  leaders,  managers,  and  evaluators  in  education  as  differentiated  from  the  Ph.D.,  which  trains 
researchers  in  the  academic  field.  They  argue  that  even  though  overlap  between  the  degrees  is  expected  and 
necessary,  the  distinction  between  the  degrees  has  blurred,  and  the  Ed.D.  is  widely  considered  a “Ph.D.  - Lite”  (p. 
27). 

Over  the  past  ten  years,  the  number  of  institutions  offering  doctoral  degree  programs  in  educational 
technology  related  fields  has  fluctuated.  Lowenthal  and  Wilson  (2008)  evaluated  the  field  by  looking  at  the  titles  of 
programs  and  degrees,  job  titles,  and  professional  organizations,  concluding  that  the  variety  of  different  labels 
impact  the  confusion  of  identity  in  the  field.  Davidson-Shiver  and  Rasmussen  (2007)  claimed,  “A  lack  of  easily 
identified,  common  titles  makes  the  field  difficult  to  distinguish  from  other  professions  in  education  and  training. 
The  lack  of  common  titles  for  IDT  programs  at  college  and  universities  makes  it  difficult  for  prospective  students  to 
match  professional  goals  to  programs.”  (p.  283). 

The  effort  to  define  the  field  of  educational  technology  is  not  new.  Gentry  (1995)  wrote,  “Regardless  of  how 
well  this  collection  of  interpretations  of  educational  technology  clarifies  or  confuses  matters,  it  is  safe  to  assume  that 
some  time  will  pass  before  precise  meanings  are  accepted  across  the  field”  (p.  8).  Reiser  (2007)  added  that  the 
definition  of  the  field  changed  over  the  years  and  there  are  no  universally  accepted  answers  to  questions  about  the 
nature  of  the  field  and  its  definition.  Persichitte  (2007 ) further  reiterated  that  the  lack  of  consensus  extends  to  the 
name  used  to  identify  the  field. 

While  there  are  sources  to  find  listings  of  doctoral  programs  in  educational  technology  and  related  fields, 
there  is  a lack  of  comprehensive  studies  to  categorize  and  evaluate  the  requirements  for  those  programs.  This  study 
provided  an  analysis  of  those  programs  and  their  curriculum.  By  taking  an  in-depth  look  at  the  similarities  and 
differences  among  the  programs,  this  study  benefits  potential  doctoral  students  seeking  appropriate  doctoral  degree 
programs  that  best  fit  their  needs.  It  also  helps  current  doctoral  students  identify  potential  universities  with  doctoral 
degree  programs  in  educational  technology  related  fields  for  their  future  employment.  In  addition,  the  study  also 
sought  to  shed  light  on  the  growth  in  doctoral  education  in  the  field  of  educational  technology  while  at  the  same 
time  evaluating  the  focus  of  the  programs. 

The  purpose  of  this  study  was  to  analyze  the  doctoral  programs  in  educational  technology  related  fields  in  the 
United  States.  The  research  addressed  the  following  questions: 

1 . What  was  the  growth  of  educational  technology  doctoral  programs  over  the  last  30  years? 

2.  How  many  institutions  (both  campus-based  and  online  programs)  offer  doctoral  degrees  in  educational 
technology? 

3.  What  is  the  variation  of  doctoral  program  degree  titles? 

4.  What  is  the  range  of  credit  hours  required  to  earn  a doctoral  degree? 

5.  What  are  the  varying  dissertation  requirements? 

6.  How  do  the  requirements  for  the  Ed.D.  differ  from  those  for  the  Ph.D.  among  universities  who  offer  both 
degrees? 


Method 


Data  Sources 

A document  analysis  method  was  implemented  for  this  study,  which  provided  key  information  about  the 
universities,  their  program  requirements,  and  curricula  central  to  addressing  the  research  questions.  By  collecting 
data  from  five  different  data  sources,  a group  of  prospective  doctoral  programs  at  varying  universities  were 
identified,  including  campus-based  and  online  programs. 

The  initial  data  source  included  the  Educational  Media  and  Technology  Yearbook  (Branch,  2008),  which 
identified  instructional  technology  related  organizations  and  graduate  programs  across  North  America.  The  second 
source  was  the  Curricula  Data  of  Degree  Programs  in  Educational  Communications  and  Technology,  an  online 
database  provided  by  the  Association  for  Educational  Communications  and  Technology  (n.d.).  This  source 
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contained  self-reported  curricula  data  of  degree  programs  in  the  educational  communications  and  technology  field 
offered  by  universities  around  the  world.  Information  provided  in  the  database  included  program  and  degree  titles, 
degrees  offered,  program  requirements,  lists  of  faculty,  and  contact  information  for  the  programs. 

In  addition  to  these  two  educational  technology  graduate  program  related  resources,  a third  data  source, 
GradSchools.com  (2009)  was  utilized.  This  online  resource  provided  a search  engine  that  allowed  us  to  discover 
educational  technology  graduate  programs  from  universities  in  the  United  States.  A fourth  data  source  consulted  for 
the  selection  of  potential  programs  was  the  online  version  of  The  College  Blue  Book  (2007).  This  electronic  resource 
included  lists  of  degrees  offered  by  universities  in  the  United  States.  The  online  version  of  The  College  Blue  Book 
provided  summaries  and  general  information  including  contact  information,  entrance  requirements,  and  website 
addresses  of  each  college  and  university  around  the  country.  The  doctoral  programs  that  had  educational  technology 
related  titles  were  selected  and  added  to  the  master  list  for  further  analysis.  Finally,  to  evaluate  each  doctoral 
program  more  completely,  we  consulted  a fifth  data  source  - the  university’s  website  for  each  of  the  doctoral 
programs  offered. 

Procedures 

We  paired  into  three  groups  for  the  first  step  in  the  data  gathering  process,  which  was  to  identify  which 
universities  in  the  United  States  offered  a Ph.D.  or  Ed.D.  program  in  educational  technology  related  fields.  One  team 
focused  on  the  Educational  Media  and  Technology  Yearbook , the  second  team  reviewed  the  AECT’s  Curricula 
Data  of  Degree  Programs  in  Educational  Communications  and  Technology  database,  and  the  third  team  collected 
data  from  Gradschools.com  and  The  College  Blue  Book.  Once  collected,  we  combined  this  data  into  a matrix  that 
listed  all  the  universities  offering  doctoral  programs  in  educational  technology  and  related  fields.  For  each  listed 
program,  we  compiled  program  names  and  degree  titles  along  with  website  addresses  and  contact  information. 

The  goal  of  using  multiple  sources  was  to  triangulate  the  program  data.  Once  we  collected  the  data,  we 
compiled  a master  list  and  removed  duplications  to  ensure  accuracy.  We  then  summarized  and  created  an  updated 
master  university  list  that  included  93  school  names,  program  website  addresses,  degree  titles,  doctoral  curriculum 
website  addresses,  type  of  degrees  (Ed.D.  or  Ph.D.).  degree  completion  requirements  (total  credit  hours  and 
dissertation  hours),  and  contact  information  from  all  four  different  sources.  Next,  we  alphabetized  the  university  list 
by  state. 

Following  this  step,  we  divided  the  list  so  each  pair  of  team  members  researched  3 1 universities  in  greater 
detail.  For  each  university,  we  identified  the  website  addresses  of  the  curriculum  for  each  program  or  its  doctoral 
program  handbook  for  later  referencing  and  cross  checking.  To  insure  validity  and  accuracy  of  this  data,  we  again 
paired  up  to  validate  the  collected  data.  Following  this  step,  we  compiled  the  curriculum  data  for  each  program.  At 
this  point,  we  evaluated  each  program  to  determine  if  the  curricula  offered  supported  educational  technology  related 
programs. 

To  conduct  this  evaluation  we  compared  the  course  names  and  curriculum  requirements  from  five  well- 
known  educational  technology  related  doctoral  programs  in  the  United  States:  Arizona  State  University,  Florida 
State  University,  Indiana  University,  Pennsylvania  State  University,  and  University  of  Georgia  (Ku,  in  press).  We 
analyzed  the  curricula  from  these  five  universities  and  developed  a set  of  curriculum  criteria.  The  curriculum  criteria 
included  (a)  foundational  courses  in  instructional  or  educational  technology  (e.g..  Introduction  to  Instructional 
Systems  or  Instructional  Design  and  Technology),  (b)  educational  practice  and  design  courses  (e.g..  Instructional 
Media  Design  or  Design  of  Distance  Educational  Environments),  (c)  cognitive  learning  and  instructional  theory 
courses  (e.g..  Theories  of  Fearning  and  Instruction  or  Theoretical  Views  of  Learning),  (d)  instructional  systems 
research  methods  or  research  design  applied  to  education  type  courses  (e.g..  Research  in  Technology,  Quantitative 
Data  Analysis,  Qualitative  Data  Analysis,  or  Mixed  Methods),  and  (e)  the  dissertation.  We  then  used  these  criteria  to 
evaluate  and  categorize  the  universities’  curricula  with  respect  to  educational  technology.  We  engaged  in  many 
conversations  to  come  to  a consensus  regarding  which  universities  had  educational  technology  or  closely  related 
programs.  The  range  of  disciplines  from  instructional  design  to  the  trends  towards  learning  sciences  perplexed  us  as 
we  assessed  each  individual  program. 

Once  we  created  a list  of  potential  university  programs,  we  evaluated  each  program’s  curriculum  for  the 
required  core  courses  and  content  to  assess  if  the  university’s  Ph.D.  or  Ed.D.  curriculum  was  substantially 
educational  technology  focused.  We  repeated  this  process  three  times,  as  we  developed  a common  understanding  of 
the  scope  of  the  field.  After  a final  comparison  of  the  degree  titles,  course  names,  and  curriculum  requirements  from 
the  principle  universities,  we  discussed  and  decided  on  which  of  the  93  universities  that  offered  doctoral  programs  in 
educational  technology  related  fields  met  the  curriculum  criteria  and  should  be  included  in  the  study. 
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Data  Analysis 

In  order  to  explore  the  growth  of  educational  technology  doctoral  programs  over  the  last  30  years,  the  records 
of  educational  technology  programs  offered  by  universities  in  the  United  States  from  five  issues  of  the  Educational 
Media  and  Technology  Yearbooks  (Witt,  1980;  Logan,  1985;  Branyan-Broadbent  & Ward,  1990;  Ely  & Minor, 
1995/1996;  Branch,  2000;  McClendon,  2005)  were  analyzed.  A descriptive  analysis  of  all  degree  titles  (Ph.D.  and 
Ed.D.)  for  both  campus-based  and  online  programs  was  conducted  with  the  goal  of  examining  the  variation  and 
frequency  of  the  different  degree  titles.  We  used  the  degree  titles  exactly  as  they  appeared  on  the  universities’ 
website  for  greater  accuracy  in  the  variation  and  frequency  analysis. 

Additionally,  the  range  of  credit  hours  required  for  the  degree  and  the  varying  dissertation  hours  required 
were  clarified  by  examining  the  specific  curricula  from  each  university.  For  programs  using  the  quarter  hour  system, 
quarter  credit  hours  were  converted  to  semester  hours  assuming  every  three  quarter  hours  is  equivalent  to  two 
semester  hours.  We  used  the  semester  hour  equivalents  for  the  comparison  of  both  course  and  dissertation 
requirements.  Lastly,  we  compared  and  evaluated  the  total  credit  hour  and  dissertation  hour  requirements  among  ten 
universities  that  offered  both  Ph.D.  and  Ed.D.  degree  programs  in  the  educational  technology  related  fields. 

Results 

The  Growth  of  the  Educational  Technology  Doctoral  Programs 

According  to  the  data  regarding  the  growth  of  educational  technology  doctoral  programs,  the  number  of 
universities  and  institutions  that  offer  doctoral  degree  programs  has  expanded  over  the  past  30  years.  The  findings 
show  that  the  number  of  doctoral  programs  in  educational  technology  related  fields  fluctuated  from  1980  to  2009. 

A total  of  42  universities  offered  doctoral  programs  in  educational  technology  related  fields  in  1980,  62  in 
1985,  64  in  1990,  47  in  1995,  49  in  2000,  54  in  2005,  and  59  in  2009,  respectively.  After  an  initial  increase  of  over 
52%  (from  42  to  64)  between  1980  and  1990,  the  number  decreased  by  23%  (from  64  to  49)  in  2000.  Then,  the 
number  of  institutions  offering  relevant  degrees  rose  over  20%  (from  49  to  59)  between  2000  and  2009. 

Numbers  of  Institutions  Offer  Doctoral  Degrees 

The  results  from  the  study  show  that  59  institutions  in  the  Unites  States  offer  doctoral  programs  in 
educational  technology  and  related  fields  in  2009.  Among  those  institutions,  55  offer  campus-based  degree 
programs  and  4 offer  online  degree  programs  in  educational  technology  and  related  fields.  Among  the  55  campus- 
based  degree  programs,  30  institutions  offer  only  Ph.D.  degree  programs,  16  institutions  offer  only  Ed.D.  degree 
programs,  and  9 institutions  offer  both  Ph.D.  and  Ed.D.  degree  programs.  In  terms  of  the  four  online  degree 
programs,  two  institutions  offer  Ph.D.  degree  programs  (Capella  University  and  Walden  University),  one  institution 
(University  of  Phoenix)  offers  an  Ed.D.  degree  program,  and  one  institution  (Northcentral  University)  offers  both 
Ph.D.  and  Ed.D.  degrees  in  educational  technology.  The  states  with  the  most  institutions  offering  educational 
technology  related  doctoral  programs  are  Florida  (6  institutions),  Pennsylvania  (4  institutions),  Texas  (4 
institutions),  and  Virginia  (4  institutions). 

The  Variation  and  Frequency  of  Doctoral  Program  Degree  Titles 

The  results  show  that  there  are  29  different  degree  titles  across  55  campus-based  universities,  resulting  in 
53%  of  the  universities  examined  offering  different  degree  titles  in  the  field  of  study  called  educational  technology. 
The  three  most  common  degree  titles  are  Instructional  Technology,  Educational  Technology,  and  Instructional 
Design  and  Technology. 

Results  for  the  four  online  university  doctoral  degree  programs  were  also  investigated  and  the  names  of  the 
degree  titles  are  Instructional  Design  for  Online  Learning,  Educational  Leadership/Educational  Technology, 
Educational  Technology  and  e-Leaming,  and  Educational  Technology.  No  common  program  degree  titles  were 
found  among  the  four  online  educational  technology  doctoral  degree  programs  reviewed.  However,  the  title 
“Educational  Technology”  is  included  in  three  (75%)  of  the  four  titles.  As  a result,  33  different  degree  titles 
represented  59  campus-based  and  online  institutions  who  offered  doctoral  degrees. 

Furthermore,  a keyword  analysis  of  both  campus-based  and  online  university  degree  titles  revealed  five 
most  commonly  occurring  words  or  word  pairs.  These  terms  by  rank  order  are  Instructional  Technology, 

Educational  Technology,  Instructional  Systems,  Curriculum  & Instruction,  and  Instructional  Design. 

The  Range  of  Total  Credit  Hours  Required 

For  the  55  campus-based  universities,  the  total  credit  hours  for  the  Ph.D.  programs  range  from  a minimum 
of  48  credit  hours  (Lehigh  University  and  University  of  Northern  Texas)  to  a maximum  of  1 13  credit  hours  (Wayne 
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State  University).  For  Ed.D.  programs,  the  total  credit  hours  range  from  42  credit  hours  (University  of  Kentucky  and 
West  Virginia  University)  to  1 13  credit  hours  (Wayne  State  University).  Two  campus-based  programs  did  not 
publish  their  total  credit  hour  requirements.  For  the  four  online  programs,  the  total  credit  hours  range  from  62 
(University  of  Phoenix)  to  89  credit  hours  (Walden  University).  Overall,  the  most  common  requirements  for  total 
credit  hours  ranged  from  56  to  65  among  59  institutions. 

The  Range  of  Dissertation  Hours  Required 

Dissertation  hours  for  the  Ph.D.  programs  range  from  a minimum  of  one  credit  hour  (University  of  Flawaii  at 
Manoa)  to  a maximum  of  30  credit  hours  (University  of  South  Florida,  Kansas  State  University,  Wayne  State 
University,  Kent  State  University,  and  Virginia  Tech  University)  with  15  credit  hours  being  the  most  common.  For 
Ed.D.  programs,  dissertation  hours  range  from  4 credit  hours  (Boston  University)  to  45  credit  hours  (University  of 
Cincinnati).  Six  campus-based  programs  did  not  publish  their  requirements  on  dissertation  hours.  The  range  of 
required  dissertation  hours  for  the  lour  online  programs  ranged  from  1 1 to  30  credit  hours.  Overall,  the  most 
common  requirements  for  dissertation  hours  ranged  from  7 to  12  among  59  institutions. 

Differences  in  Degree  Requirements  between  Ed.D.  and  Ph.D. 

Of  the  59  institutions,  10  institutions  (17%)  offered  both  the  Ph.D.  and  Ed.D.  degrees  in  educational 
technology  related  programs.  Nine  are  campus-based  (University  of  Central  Florida,  University  of  Florida,  Kansas 
State  University,  Wayne  State  University,  Pennsylvania  State  University,  University  of  North  Texas,  The  College  of 
William  & Mary,  University  of  Virginia,  and  Virginia  Tech  University)  while  one  is  an  online-based  university 
(Northcentral  University). 

Flowever,  only  nine  universities  were  compared  due  to  unavailable  data  from  Pennsylvania  State  University 
regarding  the  requirements  of  the  total  credit  hour  and  dissertation  hours.  Of  these  nine  universities,  three  (33%) 
require  their  Ph.D.  students  to  complete  more  total  credit  hours  than  Ed.D.  students.  These  three  universities  are 
University  of  North  Texas  (72  vs.  69),  The  College  of  William  & Mary  (69  vs.  48),  and  Virginia  Tech  University 
(96  vs.  90).  In  contrast,  three  universities  (33%)  require  their  Ed.D.  students  to  complete  more  total  credit  hours  than 
Ph.D.  students.  These  three  universities  are  University  of  Central  Florida  ( 108  vs.  99),  Kansas  State  University  (94 
vs.  90),  and  University  of  Virginia  (84  vs.  72).  Another  three  universities  (33%)  require  both  the  Ph.D.  and  Ed.D. 
students  to  take  the  same  amount  of  total  credit  hours.  These  three  universities  are  University  of  Florida  (90  vs.  90), 
Wayne  State  University  (113  vs.  113),  and  Northcentral  University  (51  vs.  51). 

Furthermore,  the  comparison  of  dissertation  hours  revealed  that  five  out  of  nine  universities  (56%)  require 
more  dissertation  hours  for  the  Ph.D.  program  than  for  the  Ed.D.  program.  These  five  universities  are  University  of 
Central  Florida  (24  vs.  21),  Kansas  State  University  (30  vs.  16),  Wayne  State  University  (30  vs.  20),  The  College  of 
William  & Mary  (9  vs.  6),  and  Virginia  Tech  University  (30  vs.  24).  Four  universities  (44%)  require  the  same 
dissertation  hours  for  both  degrees.  These  four  universities  are  University  of  Florida  ( 12  vs.  12),  University  of  North 
Texas  (12  vs.  12),  University  of  Virginia  (12  vs.  12),  and  Northcentral  University  (9  vs.  9). 

Discussion 

According  to  our  findings,  educational  technology  doctoral  programs  increased  from  42  in  1980  to  64  in 
1990.  This  increase  parallels  the  development  of  advanced  technologies  in  the  United  States  after  1980.  As  Senese 
(1983)  stated,  the  beginning  of  the  technological  revolution  in  1980  affected  both  worlds  of  work  and  education. 
Technology  became  a significant  component  for  improving  the  educational  system.  Many  teachers  and  instructors 
were  encouraged  to  include  technology  within  their  instruction  to  facilitate  student  learning.  Teachers  and 
instructors  were  required  to  have  basic  skills  in  technology  usage.  Therefore,  the  increasing  need  for  combining 
teaching  and  technology  in  education  became  a major  influence  on  the  number  of  educational  technology  related 
programs  offered  from  1980  to  1990.  However,  the  number  of  institutions  offering  educational  technology  doctoral 
degrees  declined  from  64  in  1990  to  49  in  2000.  This  phenomenon  may  have  paralleled  a significant  economic 
downturn  in  the  United  States  where  budgets  for  improving  and  expanding  programs  were  limited  in  both  public  and 
private  universities  in  the  1990s.  Therefore,  several  programs  within  institutions  were  closed,  which  led  to  a 
decreasing  number  of  programs  in  the  field. 

As  the  economy  improved  and  significant  advances  were  made  in  technology,  within  a few  years,  the 
number  of  institutions  offering  educational  technology  and  related  degrees  increased  from  49  in  2000  to  59  in  2009. 
One  major  impetus  to  support  this  increase  may  be  the  rising  numbers  of  distance  learning  courses  offered  by 
various  institutions.  Feonard  and  Guha  (2001)  shared  that  online  courses  presented  flexible  options  allowing 
students  to  access  classes  anytime  and  anywhere,  which  responded  to  the  needs  of  students  with  busy  lifestyles  and 
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second-career  college  students  who  previously  faced  a barrier  to  attending  regular  classes  on  campus.  Dobbs,  Waid, 
and  Carman  (2009)  also  pointed  out  that  the  affordability  and  accessibility  of  technology  for  students,  especially 
computer  and  Internet  accessibility,  led  to  an  exponentially  growing  demand  for  online  courses. 

There  is  a dramatic  difference  between  the  number  of  campus-based  degree  programs  and  online  degree 
programs.  Fifty-five  institutions  (93%)  are  campus-based  programs  while  four  (7%)  institutions  offer  online  doctoral 
degrees.  This  may  indicate  that  the  demand  for  online  doctoral  degree  programs  is  not  as  strong  as  for  campus-based 
programs.  It  may  also  point  to  issues  around  meeting  residency  doctoral  degree  requirements  and  the  challenges  of 
pursuing  a doctoral  degree  with  limited  face  to  face  instructor  contact  and  cohort  contact.  Furthermore,  students  and 
faculty  may  perceive  the  quality  of  learning  found  in  campus-based  programs  to  be  better  than  online -based  degree 
programs.  In  addition,  Adams  and  DeFleur  (2006)  indicated  that  employers  prefer  traditional  degree  programs  to 
online  degree  programs.  This  may  also  have  an  impact  on  the  demand  for  online  programs. 

The  variance  and  frequency  of  the  33  different  degree  titles  serve  to  underpin  the  lack  of  a clear  definition 
and  consistency  within  the  field  of  educational  or  instructional  technology.  However,  it  seems  logical  that  as  the 
field  has  varied  definitions  and  interpretations,  so  do  the  programs  and  degree  titles  that  prepare  professionals  for 
entry  into  the  field.  The  fact  that  the  degree  title  Instructional  Technology  occurs  12  times  among  the  59  doctoral 
programs  supports  what  Reiser  (2007)  claimed  to  be  the  term  used  most  frequently  to  describe  our  field. 

One  explanation  for  the  variety  of  program  titles  in  our  field  is  that  the  program  degree  titles  may  be  driven 
by  a university’s  marketing  initiatives.  In  an  effort  to  attract  students,  universities  need  to  differentiate  themselves  to 
make  themselves  stand  out  and  appeal'  unique  in  the  eyes  of  prospective  students.  One  way  to  do  that  is  to  create  a 
program  name  that  is  different,  catchy,  or  sounds  innovative,  such  as  learning  sciences  or  learning  design  and 
technology.  Conversely,  the  absence  of  the  words  instructional  design  from  a program  title  may  mean  that  a key 
component  of  our  field,  instructional  design,  is  not  as  heavily  emphasized  by  that  particular  curriculum.  As  Merrill 
pointed  out  in  his  debate  with  Wilson  (Merrill  & Wilson,  2007),  down  the  broad  road  of  instructional  technology,  it 
may  be  more  and  more  difficult  to  determine  if  there  is  an  academic  field  of  instructional  design  (p.  349).  Either 
way,  how  a university  positions  and  markets  its  educational  or  instructional  technology  doctoral  program  may 
influence  the  degree  title  and  program  curricula. 

This  study  provided  an  overall  view  of  the  credit  hour  range  for  completing  a Ph.D.  or  Ed.D.  degree  in 
educational  technology  related  fields  in  the  United  States.  According  to  Wellman  (2005),  institutions  have  a high 
degree  of  flexibility  to  vary  credit  hour  requirements  to  meet  internal  institutional  standards.  The  variation  of  the 
credit  hours  for  the  Ph.D.  (48-113  semester  hours)  and  Ed.D.  programs  (42-113  semester  hours)  in  the  educational 
technology  field  is  extensive.  In  terms  of  dissertation  hours,  the  range  and  variability  for  the  number  of  dissertation 
hours  required  by  various  programs  reflects  the  range  and  variability  found  in  degree  titles.  One  reason  for  this 
variability  could  be  due  to  the  challenges  faced  by  doctoral  education  in  the  United  States  in  general.  According  to 
Nerad  (2008),  one  of  the  challenges  faced  by  U.S.  doctoral  education  includes  managing  the  intricate  link  between 
doctoral  education  and  the  institutional  research  mission  of  the  university.  Similar  to  the  total  credit  hours,  the  range 
of  the  total  dissertation  hours  varies  depending  on  the  program’s  goals  and  mission. 

Finally,  this  study  addressed  the  different  requirements  for  the  Ph.D.  and  Ed.D.  programs  in  the  educational 
technology  related  fields.  Several  sources  indicated  that  both  the  Ph.D.  and  Ed.D.  programs  were  designed  for 
different  purposes  and,  therefore,  prepared  students  differently  (Shulman,  Golde,  Bueschel,  & Garabedian,  2006; 
Deering,  1998.)  Of  the  nine  programs  that  offer  both  Ph.D.  and  Ed.D.  degrees,  three  programs  require  more  total 
credit  hours  for  the  Ph.D.  than  the  Ed.D.  degrees,  three  universities  require  the  same  number  of  total  credit  hours  for 
both  degrees,  and  three  programs  require  less  total  credit  hours  for  the  Ph.D.  than  the  Ed.D.  degrees.  Therefore, 
there  is  no  significant  relationship  between  total  credit  hours  required  of  the  Ph.D.  and  the  Ed.D.  programs. 

Although  Shulman  et  al.  (2006)  declared  that  the  Ed.D.  is  often  viewed  as  a Ph.D.  Lite,  “with  fewer  requirements 
than  the  Ph.D.,  much  less  emphasis  on  fulltime  study  and  residency...”  (p.  27),  our  findings  did  not  demonstrate  a 
disparity  of  effort  between  degree  types. 

Even  though  the  Ph.D.  and  the  Ed.D.  programs  serve  different  theoretical  goals,  they  are  “similar  in 
programmatic  requirements,  knowledge  bases,  competency  standards,  and  in  employment  expectancies”  (Andersen, 
1983,  p.  5).  In  terms  of  dissertation  hours,  the  results  revealed  that  the  Ph.D.  degree  required  more  dissertation  hours 
than  the  Ed.D.  degree.  Deering  (1998)  stated  that  the  Ph.D.  dissertation  illustrated  scholarly  and  high  quality 
research  and  the  Ed.D.  dissertation  was  more  applied  but  not  theoretical  in  nature.  Redden  (2007)  also  implied  that  a 
Ph.D.  dissertation  is  more  research-oriented  while  an  Ed.D.  dissertation  is  more  applied  and  practitioner-oriented. 
Therefore,  institutions  who  offer  both  Ph.D.  and  Ed.D.  degrees  in  the  same  field  might  demand  more  dissertation 
hours  for  Ph.D.  than  Ed.D.  degrees. 

As  with  many  studies,  there  are  limitations  as  well  as  areas  for  future  investigation.  This  study  does  not 
explain  or  detail  what  the  degree  program  fluctuations  represented  beyond  the  growth  or  decline  of  numbers.  Further 


287 


investigation  may  shed  light  about  which  degree  titles  have  remained  stable,  which  are  in  decline  and  which  are 
new.  In  addition,  future  research  can  focus  on  in-depth  curriculum  analysis  among  these  doctoral  programs  to  help 
clarify  the  core  competencies  in  the  field  of  educational  technology. 
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Descriptors:  Geospatial  information  technologies,  Instructional  design 

Geospatial  information  technologies  such  as  geographic  information  systems  (GIS),  global  positioning 
systems  (GPS),  and  global  visualization  tools  (for  example,  Google  Earth,  and  WorldWind)  are  increasingly  being 
incorporated  in  the  classrooms  to  promote  student  problem  solving,  data  analysis,  and  critical  thinking.  These  tools 
allow  for  visualizing,  mapping,  organizing,  and  analyzing  multiple  layers  of  geographically  referenced  information 
(Broda  & Baxter,  2002;  DeMers,  2005;  ESRI,  1993).  Baker  and  Case  (2000)  suggested  GIS  may  be  a promising 
educational  technology  for  developing  contextually  rich  student  learning  and  Kerski  (2008)  noted  the  use  of  GIS  in 
education  is  increasingly  viewed  as  facilitating  active  learning  to  engage  students  in  critical  thinking  (see  also  Bull 
& Mason,  1998;  Ramirez,  1995;  Sanders,  Kajs  & Crawford,  2002).  Bednarz  and  Audet  (1999)  agreed,  suggesting  a 
GIS  may  provide  an  authentic,  inquiry-based  learning  environment  for  the  K-12  classroom.  In  fact,  K-12  educators 
are  harnessing  the  power  of  GIS  technology  to  support  standards-based  math,  science,  geography,  and  social  studies 
curricula  (Holzberg,  2006).  There  is,  however,  a paucity  of  research  on  implementing  GIS  in  science  classrooms. 

Keiper  (1998)  identified  as  barriers  to  implementing  a GIS  in  the  classroom  lack  of  a specific  relevant 
curriculum  that  includes  a GIS,  student  frustration  with  the  technology,  and  teachers’  lack  of  a pedagogical  style 
conducive  to  using  a GIS  (see  also  Sanders  et  al.,  2002).  According  to  Baker  and  Case  (2000),  many  teachers  find 
time  a limiting  factor  in  using  a GIS  because  they  need  time  on  their  own  to  learn  and  practice  using  the  GIS  and 
then  need  quite  a bit  of  time  to  teach  students  to  use  the  software.  It  appears  using  a GIS  in  the  classroom  may  make 
some  unusual  cognitive  demands  on  the  learners  and  demands  on  teachers’  limited  instructional  time. 

Purpose  of  the  Study 

Given  the  paucity  of  research  on  implementing  GIS  in  science  classrooms  and  the  promising  nature  of  GIS 
to  foster  science  inquiry,  our  goal  was  to  determine  how  a Web-based  module  might  best  be  designed  and  developed 
to  enhance  science  inquiry  supported  by  GIS  with  classes  of  eighth-grade  students. 

Design  Conditions  and  Limitations 

There  were  five  conditions  and  limitations  that  shaped  how  we  designed  our  materials  and  what 
instructional  approaches  we  are  able  to  employ. 

1.  The  students  in  our  study  were  of  diverse  ethnic  backgrounds  with  19%  of  them  being  non-native  English 
speakers.  Eighty-three  students  (41%)  received  lower  scores  on  the  2008  state  standardized  reading  skills 
test.  Thus,  a large  portion  of  our  sample  would  be  classified  as  low-level  readers. 

2.  One  hundred  and  ninety-eight  students  (78%)  received  lower  scores  on  the  2008  state  standardized  test  of 
science  knowledge.  Thus,  more  than  three  quarters  of  these  students  would  be  classified  as  low-level 
science  learners. 

3.  The  school  had  adopted  the  Understanding  by  Design  instructional  development  process  model  for  its  new 
science  curriculum.  We  needed,  therefore,  to  make  sure  that  whatever  we  designed  also  conformed  to  the 
requirements  of  this  model. 

4.  The  science  content  was  developed  by  science  education  experts  and  earth  and  environmental  science 
experts  who  were  part  of  the  larger  design  team.  As  with  all  design  projects,  the  instructional  designer’s 
role  was  to  organize,  sequence,  and  design  science  instruction,  not  the  content.  The  instruction  was  to  be 
completed  in  eight  weeks  and  all  developed  materials  had  to  be  reviewed  and  approved  by  the  design  team. 

5.  The  instructional  unit  was  to  be  supported  by  My  World  GIS.  The  students  and  the  teacher  did  not  have 
experience  using  this  particular  GIS  software.  They  were  going  to  use  My  World  GIS  for  the  first  time. 
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Selected  Instructional  Design  Theories 


The  theories  that  guided  us  in  the  design  process  include  behaviorism,  constructivism,  and  inquiry  teaching. 
We  chose  the  behaviorist  theory  because  the  population  of  students  with  which  the  study  was  to  be  conducted  is  one 
that  research  suggests  is  likely  to  benefit  from  direct  instruction  and  extensive  practice,  two  behaviorist  approaches 
(Gersten,  Keating,  & Becker,  1988).  In  addition,  behaviorists  tend  to  emphasize  correct  responses  and  we  were 
concerned  that  the  students  in  our  study  would  be  measured  against  state  standardized  tests  that  employ  behaviorist 
assessment  techniques  almost  exclusively.  We  employed  many  instructional  practices  classified  as  behaviorist  (for 
syntheses  of  such  practices,  see  Bugelski,  1971  and  Snelbecker,  1974).  We  chose  the  constructivist  theory  because 
we  wanted  students  to  modify  and  refine  what  they  already  know  and  collaborate  on  performing  tasks  as  they 
learned  from  one  another.  We  used  many  instructional  practices  deriving  from  constructivism  (for  syntheses,  see 
Bednar,  Cunningham,  Duffy  & Perry,  1992;  Duffy  & Jonassen,  1992;  Honebein,  1996;  Knuth  & Cunningham, 

1993;  and  Savery  & Duffy,  1996).  We  chose  Collins  and  Stevens’  (1983)  expression  of  the  theory  of  inquiry 
teaching  because  we  wanted  students  to  engage  with  hands-on  activities  and  use  evidence  to  justify  their 
explanations  just  like  scientists  do.  Further,  scientific  inquiry  is  the  central  tenet  of  science  education  reform. 

Selected  Instructional  Models 

We  used  four  instructional  development  process  models  and  three  instructional  design  models  to  design 
instruction.  Instructional  development  process  models  describe  the  phases/stages  of  designing  and  developing 
instruction,  while  instructional  design  models  provide  steps  to  guide  designers  organize,  structure,  and  sequence 
instruction.  Designers  use  instructional  design  models  to  make  decisions  about  how  content  should  be  sequenced, 
how  to  use  overviews  and  examples,  how  to  practice,  and  how  to  assess  learning. 

Instructional  Development  Process  Models 

The  first  instructional  development  model  we  chose  was  the  generic  ADDIE  model.  ADDIE  is  an  acronym 
for  Analyze,  Design,  Develop,  Implement,  and  Evaluate  (Learning  Theories  Knowledgebase,  2008).  We  classified 
the  other  three  models  according  to  how  they  mapped  to  the  ADDIE  model.  The  second  model  was  the  Dick,  Carey, 
and  Carey  (2005)  model  which  breaks  ADDIE’s  basic  five  phases  into  nine  stages.  A phase  may  contain  one  or 
more  stages  and  stages  break  down  phases  into  more  specific  actions  and  provide  more  guidance  on  how  to  design. 
Further,  the  Dick,  Carey  and  Carey  model  makes  clear  that  the  process  is  iterative;  that  is,  a designer  goes  back  to 
any  of  the  stages  of  design  or  development  to  make  revisions.  Although  the  model  specifies  sequenced  stages,  it 
recognizes  explicitly  that  many  design  processes  are  complementary  and  may  occur  near  simultaneously.  The  third 
model  we  used  was  Wiggins  and  McTighe’s  (1998)  Understanding  by  Design  model  that  focuses  on  a topic’s 
enduring  understandings.  Enduring  understandings  refer  to  the  big  and  important  ideas  that  the  teacher  wants  the 
students  to  comprehend  and  retain.  The  authors  contended  that  “by  having  students  encounter  big  ideas  in  ways  that 
provoke  and  connect  to  students’  interests,  we  increase  the  likelihood  of  student  engagement  and  sustained  inquiry” 
(p.  11).  Our  fourth  model  was  Keller’s  (1983)  ARCS  motivation  model  that  focuses  on  enhancing  student 
motivation.  ARCS  is  an  acronym  for  Attention,  Relevance,  Confidence,  and  Satisfaction.  Given  that  the  learners 
were  low-level  science  learners,  sustaining  their  motivation  was  crucial  for  their  success  in  this  unit. 

Instructional  Design  Models 

We  chose  three  instructional  design  models  to  match  the  theories  that  we  used.  For  the  behaviorist 
instructional  model,  we  selected  Gagne’s  (1977)  Nine  Significant  Events  model  because  it  represents  a sequence 
many  effective  teachers  use  in  direct  instruction.  A constructivist  model  seemed  well  suited  to  this  project  because 
of  the  open-ended  nature  of  the  learning  tasks  and  activities.  We  chose  two  constructivist  design  models:  Jonassen’s 
(1999)  model  for  designing  constructivist  learning  environments  and  Black  and  McClintock’s  (1996)  interpretation 
construction  (ICON)  model.  These  two  models  appeared  well  matched  to  the  content  and  target  audience.  Under 
inquiry  teaching,  we  used  three  inquiry  models:  Bybee  et  al’s  (1989)  5E  model  that  has  been  used  over  the  years  to 
design  instruction;  Eisenkraft’s  (2003)  7E  model  that  adds  two  steps  to  the  5E  model;  and  the  National  Research 
Council’s  ([NRC],  2000)  Five  Essential  Features  model  that  specifically  applies  to  science. 

Derived  Instructional  Model 

As  a result  of  our  analyses  of  the  various  design  models,  we  derived  an  instructional  model  and  three  sub- 
models for  teachers  to  follow  and  we  developed  materials  that  fit  our  derived  model  and  its  sub-models.  The  larger 
instructional  model  has  four  major  steps,  the  second  of  which  represents  presentation  of  all  types  of  instructional 
content  and  incorporates  our  three  sub-models.  The  three  sub-models  are  unified  under  the  larger  model  and  they 
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present  models  for  the  presentation  of  content,  for  computer-supported  activities,  and  for  laboratory  activities.  The 
steps  of  our  instructional  model  (and  sub-models)  are  listed  below  and  explained  in  the  paragraphs  that  follow. 

1 . Confirm  learners  have  necessary  background. 

1 . 1 Administer  content  knowledge  pretest. 

1 .2  Administer  attitude  towards  science  and  technology  pretest. 

1.3  Elicit  and  discuss  prior  understandings  of  unit  concepts  aloud. 

1.4  Elicit  additions  to  the  concept  map  independently. 

1.5  Identify  misconceptions  from  student  responses. 

2.  Present  instruction  using  the  appropriate  model. 

2.1  Instructional  model  for  content  presentation 

2.1.1  Elicit  prior  understandings  of  lesson  concepts. 

2.1.2  Gain  and  sustain  learners’  attention. 

2.1.3  Tell  learners  the  objectives. 

2.1.4  Stimulate  recall  of  prerequisite  learning. 

2.1.5  Explain  content. 

2.1.6  Illustrate  content. 

2.1.7  Elicit  answers  to  specific  questions  on  students’  worksheets. 

2.1.8  Solicit  some  responses  from  students’  worksheets  and  provide  feedback  aloud. 

2.1.9  Review  content. 

2.2  Instructional  model  for  computer-supported  activities 

2.2.1  Elicit  prior  understandings  of  lesson  concepts. 

2.2.2  Present  authentic  task. 

2.2.3  Model  task. 

2.2.4  Provide  worked  example. 

2.2.5  Ask  learners  to  perform  task. 

2.2.6  Scaffold  task. 

2.2.7  Ask  learners  additional  questions  to  elaborate  task. 

2.2.8  Review  activity  concepts. 

2.3  Instructional  model  for  laboratory  activities 

2.3.1  Elicit  prior  understandings  of  lesson  concepts. 

2.3.2  Present  authentic  task. 

2.3.3  Form  student  groups. 

2.3.4  Model  task. 

2.3.5  Ask  students  to  make  predictions. 

2.3.6  Ask  group  members  to  collaborate  on  task. 

2.3.7  Elave  students  make  observations. 

2.3.8  Elave  students  use  evidence  to  form  explanations. 

2.3.9  Elave  students  evaluate  explanations  and  draw  conclusions. 

2.3.10  Elave  students  share  and  justify  results. 

2.3.11  Address  misconceptions. 

2.3.12  Ask  learners  to  perform  extension  tasks. 

2.3.13  Review  activity  concepts. 

3.  Confirm  instruction  is  meeting  goals  and  objectives. 

3.1  Ask  questions  aloud  and  respond  to  student  answers. 

3.2  Solicit  and  respond  to  student  questions. 

3.3  Check  students’  worksheet  responses  aloud. 

3.4  Provide  feedback  aloud. 

3.5  Elave  students  add  new  ideas  to  their  concept  maps. 

3.6  Adjust  instruction  to  meet  learners’  needs. 

4.  Confirm  learners  have  acquired  desired  knowledge,  skills,  and  attitudes. 

4.1  Assess  culminating  activity. 

4.2  Assess  concept  map. 

4.3  Administer  and  analyze  content  knowledge  posttest. 

4.4  Administer  and  analyze  attitude  towards  science  and  technology  posttest. 
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Step  1 is  based  on  Dick,  Carey  and  Carey’s  “identifying  and  analyzing  entry  behaviors  and  learner 
characteristics,”  Eisenkraft’s  first  E,  elicit,  and  Jonassen’s  constructivist  step  of  providing  knowledge  representation 
tools. 

In  Steps  1.1  and  1.2,  the  teacher  administers  a content  knowledge  and  attitude  and  behavior  pretests  to 
determine  what  knowledge,  skills,  and  behaviors  students  bring  to  the  learning  task. 

In  Step  1.3,  the  teacher  uses  oral  questions  to  identify  what  students  know  about  energy  (for  example, 

“What  is  energy?”  or  “Where  does  energy  come  from?”).  The  teacher  then  discusses  learners’  responses. 

In  Step  1.4,  the  teacher  asks  students  to  brainstorm  on  their  own,  listing  everything  they  know  about  energy 
and  then  to  add  to  their  individual  concept  maps.  Students  add  ideas  to  their  concept  maps  periodically  as  they 
complete  the  unit.  The  goal  is  for  learners  to  construct  their  own  relationships  among  concepts  as  they  learn  them. 

In  Step  1.5,  the  teacher  addresses  misconceptions  learners  may  have  about  unit  concepts.  It  is  likely  the 
teacher  will  examine  student  responses  to  the  pretests  outside  of  class  to  check  for  students’  misconceptions  and 
decide  which  areas  to  address  and  clarify  when  teaching. 

As  noted  earlier,  Step  2 encompasses  three  sub-models;  the  first  for  presenting  content,  the  second  for 
doing  computer-supported  activities,  and  the  third  for  doing  laboratory  activities.  The  instructional  sub-model  for 
content  presentation  comes  first  because  students  need  to  acquire  and  understand  science  content  before  they  can 
practice.  The  instructional  sub-model  for  the  computer-supported  activities  is  presented  second  because  the  unit  is 
primarily  based  on  using  GIS  to  support  science  teaching  and  learning.  Computer-supported  activities  include  using 
the  GIS  software,  Google  Earth,  and  spreadsheets  to  perform  tasks.  The  content  and  activities  are  organized  around 
Wiggins  and  McTighe’s  (2005)  framework  of  focusing  on  big  ideas.  Laboratory  activities  augment  GIS  activities. 
Teachers  employ  the  appropriate  sub-model  -or  a combination  of  sub-models-  depending  on  the  nature  of  the 
activity  for  a class.  The  three  instructional  sub-models  are  discussed  below. 

Instructional  model  for  presenting  content 

Step  2.1.1  matches  Dick,  Carey  and  Carey’s  “identifying  and  analyzing  entry  behaviors  and  learner 
characteristics,”  and  Eisenkraft’s  first  E,  elicit.  The  teacher  asks  students  questions  about  the  specific  lesson 
concepts  and,  in  this  way,  identifies  what  knowledge  and  skills  learners  bring  to  the  learning  task.  For  example,  a 
teacher  might  ask,  “What  is  solar  energy?”  when  introducing  students  to  solar  energy. 

Step  2.1.2  combines  Bybee  et  al.’s  first  E,  engage,  Gagne’s  gain  attention,  and  Keller’s  attention.  In  the 
unit,  we  use  brief  videos,  animations,  demonstrations,  oral  stories,  and  objects  to  capture  attention.  For  example,  the 
teacher  begins  the  lesson  on  geothermal  energy  by  showing  students  a brief  video  of  geothermal  areas  in  Iceland. 
Throughout  lessons,  this  sub-model  uses  such  ARCS  (Keller,  1983)  attention-sustaining  strategies  as  posing 
questions,  engaging  learners  with  tasks,  and  balancing  content  presentation  with  interactive  sessions. 

Step  2.1.3  matches  Gagne’s  “inform  the  learner  of  objectives”  with  Keller’s  relevance  by  presenting 
objectives  in  a way  that  conveys  the  usefulness  of  the  instruction  (for  example,  by  telling  teachers  such  things  as, 
“Inform  students  that  they  will  investigate  ways  of  conserving  energy.” 

Step  2.1.4  is  drawn  verbatim  from  Gagne’s  Nine  Significant  Events  model.  The  teacher  reminds  students  of 
important  prerequisite  knowledge  or  skills  they  have  learned  previously.  This  also  helps  prepare  learners  for  the  new 
content. 

Step  2.1.5:  Using  direct  instruction,  demonstrations,  and  videos,  the  teacher  explains  the  new  content  or 
students  access  new  content  on  the  unit’s  student  resources  Web  page.  This  step  integrates  Bybee  et  al.’s  third  E 
(explain)  and  Gagne’s  “present  stimuli.”  The  teacher  uses  small  cycles  of  events  to  present  the  content  and  images 
are  used  in  the  materials  to  reduce  the  reading  load  and  provide  some  content  redundancy. 

Step  2.1.6  interweaves  Collins  and  Stevens’  inquiry  teaching  strategies,  Gagne’s  “provide  guidance,”  and 
Keller’s  relevance.  Examples,  illustrations,  and  answers  to  student  questions  are  used  to  help  learners  understand 
new  content.  The  teacher  relates  examples  to  the  learner’s  experience  and  values  and  uses  positive  and  negative 
examples  and  counterexamples  as  well.  For  example,  students  receive  teacher  guidance  on  how  to  complete  the 
personal  energy  audit  in  which  they  note  their  daily  and  weekly  energy  consumption. 

Step  2.1.7  is  an  adaptation  of  Gagne’s  “elicit  performance”  event.  Teachers  ask  learners  to  respond  on  their 
worksheets  to  specific  questions  as  the  content  is  presented. 

Step  2.1.8:  This  is  Gagne’s  event  seven,  “provide  feedback,”  although  we  have  the  teacher  provide  such 
feedback  aloud.  The  teacher  asks  students  to  share  their  worksheet  responses  with  the  class  and  he  or  she  then 
discusses  some  of  those  responses  with  the  class.  In  this  way,  the  teacher  reinforces  correct  responses,  clarifies 
misunderstandings,  and  summarizes  the  content.  This  step  also  addresses  Keller’s  satisfaction.  Teacher-provided 
feedback  should  help  to  sustain  desired  behavior. 
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In  Step  2.1.9,  lesson  concepts  are  reviewed  by  the  teacher  in  order  to  reinforce  student  learning  and  to 
clarify  any  concepts  students  did  not  understand.  This  is  a variation  of  Gagne’s  “enhance  retention  and  transfer.” 
Instructional  model  for  computer-supported  activities 

Step  2.2.1  combines  Eisenkraft’s  first  E,  elicit  and  Dick,  Carey  and  Carey’s  “identifying  and  analyzing 
entry  behaviors  and  learner  characteristics.”  The  teacher  asks  questions  about  lesson  concepts  to  determine  what 
knowledge  and  skills  students  bring  to  the  learning  task. 

In  Step  2.2.2,  the  students  become  aware  of  their  authentic  task  when  the  teacher  presents  it.  This  is  a 
modification  of  Jonassen’s  “select  an  appropriate  task  for  learners  to  do.”  Instructional  materials  present  tasks  in 
different  ways.  In  some  tasks,  students  analyze  regional  or  worldwide  cases  first  and  then  move  to  local  cases.  In 
other  tasks,  students  analyze  in  the  opposite  direction.  Regardless,  this  step  obeys  Collins  and  Stevens’  strategy  to 
vary  cases  systematically. 

Step  2.2.3:  Teachers  model  for  learners  how  to  do  the  task,  for  example,  how  to  obtain  data  about  solar 
power  plants  using  the  My  World  GIS  Get  Information  tool.  This  is  an  instance  of  both  Jonassen’s  and  Black  and 
McClintock’s  steps  in  which  the  teacher  models  the  task. 

Step  2.2.4  implements  Jonassen’s  “provide  worked  examples.”  Either  the  teacher  or  the  materials  (or  both) 
provides  a worked  example  to  help  guide  students  in  performing  the  task.  For  example,  in  our  energy  unit,  the 
materials  provide  a worked  example  of  how  students  should  complete  the  solar  power  plants  data  chart.  As  noted 
earlier,  the  materials  also  provide  positive  and  negative  examples,  and  counterexamples  (Collins  & Stevens,  1983) 
to  highlight  important  things  that  should  help  learners  complete  the  task.  For  instance,  the  materials  provide 
screenshots  of  what  students  would  see  when  they  perform  a task  either  correctly  or  incorrectly. 

Step  2.2.5:  Learners  perform  the  task.  This  step  integrates  the  NRC’s  first  essential  feature  (“learners 
engage  with  a scientifically  oriented  question”),  Keller’s  satisfaction,  and  Bybee  et  al.’s  second  E,  explore.  Learners 
construct  their  own  understandings  by  engaging  actively  with  the  task.  To  address  Keller’s  satisfaction,  teachers 
have  learners  use  their  newly  acquired  knowledge  and  skills  to  manipulate  data  in  a simulated  setting.  For  instance, 
the  culminating  task  has  students  apply  the  unit’s  knowledge  and  skills  to  recommend  the  best  combination  of 
energy  sources  for  a fictional  island. 

In  Step  2.2.6,  the  materials  and  the  teacher  provide  guidance  to  the  learners  as  they  complete  GIS  tasks. 

This  honors  Jonassen’s  steps  where  the  teacher  coaches  the  learners  and  provides  cognitive  tools  to  support  the 
learners’  performance.  In  our  instructional  unit,  learners  only  use  the  GIS  when  they  need  it  to  accomplish  a learning 
task.  An  orientation  to  the  GIS  is  given  by  the  teacher  and  he  or  she  models  how  to  use  it  to  visualize,  manipulate, 
and  analyze  data.  Students  leam  to  use  the  GIS  through  completion  of  a series  of  authentic  tasks.  We  have 
scaffolded  the  handouts  for  using  GIS  heavily:  They  use  screenshots,  hints,  and  a consistent  sequence  to  help  enable 
students  to  use  them  to  complete  tasks  independently  outside  the  classroom.  GIS  activities  are  integrated  with  non- 
GIS  activities  that  learners  already  know  how  to  do  so  that  students  do  not  become  overwhelmed  as  they  begin 
working  with  the  GIS  software  with  which  they  have  no  prior  experience. 

Step  2.2.7  addresses  Bybee  et  al.’s  fourth  E,  elaborate.  The  teacher  and  materials  ask  students  to  answer 
higher-order  questions,  draw  conclusions,  and  reflect  on  how  concepts  relate  to  each  other.  All  this  is  done  in  hopes 
of  fostering  greater  student  understanding. 

Step  2.2.8  is  a variation  on  Gagne’s  “enhance  retention  and  transfer.”  To  reinforce  student  learning,  the 
teacher  reviews  the  concepts  learned  in  the  activity.  This  review  should  also  allow  the  teacher  to  clarify  any 
concepts  students  did  not  understand. 

Instructional  model  for  laboratory  activities 

Step  2.3.1  combines  Eisenkraft’s  first  E,  elicit  and  Dick,  Carey  and  Carey’s  “identifying  and  analyzing 
entry  behaviors  and  learner  characteristics.”  Just  before  beginning  the  activity,  students  are  asked  questions  about 
lesson  concepts  in  order  to  identify  what  knowledge  and  skills  they  bring  to  the  learning  task. 

Step  2.3.2:  The  teacher  presents  the  task  to  the  learners.  This  is  a modification  of  Jonassen’s  “select  an 
appropriate  task  for  learners  to  do.” 

In  Step  2.3.3,  the  teacher  assigns  students  to  groups  in  which  they  will  perform  laboratory  experiments. 

Step  2.3.4:  The  teacher  demonstrates  the  task.  This  represents  both  Jonassen’s  and  Black  and  McClintock’s 
teacher  modeling  of  the  task. 

Step  2.3.5  implements  Collins  and  Stevens’  “make  predictions”  by  having  the  teacher  ask  students  to  make 
predictions  before  they  begin  the  laboratory  experiment. 

Learners  conduct  laboratory  experiments  in  Step  2.3.6.  This  step  integrates  the  NRC’s  first  essential  feature 
(“learners  engage  with  a scientifically  oriented  question”)  with  Bybee  et  al.’s  second  E,  explore,  and  having  students 
work  collaboratively  on  tasks  (Black  & McClintock,  1996;  Jonassen,  1999). 
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In  Step  2.3.7,  learners  make  observations  of  their  laboratory  experiments,  thus  implementing  Black  and 
McClintock’s  student  observations  of  authentic  artifacts. 

Step  2.3.8:  Obeying  the  NRC’s  third  essential  feature  (“learners  to  give  priority  to  evidence”)  and  Collins 
and  Stevens’  “consider  alternative  predictions,”  students  focus  on  evidence,  think  about  alternative  predictions,  and 
formulate  explanations  from  evidence.  In  keeping  with  Bybee  et  al.’s  third  E,  explain,  students  discuss  explanations 
and  alternatives  and  expand  their  understanding  of  concepts. 

In  Step  2.3.9  learners  evaluate  their  explanations  in  light  of  alternative  explanations,  and  think  their  way 
towards  solutions.  This  step  integrates  Bybee  et  al.’s  fifth  E,  evaluate,  the  NRC’s  fourth  essential  feature  (“ask 
learners  to  evaluate  their  explanations  in  light  of  alternative  explanations”),  particularly  those  reflecting  scientific 
understanding,  and  Collins  and  Stevens’  “question  authority”  (modified  here  to  “draw  own  conclusions”). 

Step  2.3.10  is  based  on  the  NRC’s  fifth  essential  feature  (“learners  communicate  and  justify  their  proposed 
explanations”)  and  Black  and  McClintock’s  steps.  Learners  share  results  and  present  and  defend  their  explanations. 
We  anticipate  that  this  should  enhance  their  motivation  to  be  well  prepared  and  may  help  to  give  them  a sense  of 
ownership  of  the  content.  This  step  is  also  designed  to  address  Keller’s  confidence.  In  formulating  an  energy  policy 
for  a fictional  island,  students  are  provided  with  performance  requirements  and  evaluative  criteria  to  support  them  as 
they  complete  the  task. 

Step  2.3.11  represents  Collins  and  Stevens’  “trace  consequences  to  a contradiction”  and  in  this  step  the 
teacher  corrects  student  misconceptions  evinced  in  students’  explanations. 

Step  2.3.12  is  based  on  Eisenkraft’s  seventh  E,  extend.  Teachers  ask  learners  to  use  the  skills  they  have 
acquired  to  perform  additional  tasks.  The  intent  is  to  enhance  the  transfer  of  student  learning. 

Step  2.3.13  reflects  Gagne’s  ninth  event  (“enhance  retention  and  transfer”).  To  reinforce  student  learning 
and  clarify  any  concepts  about  which  students  seem  confused,  the  teacher  reviews  the  concepts  covered. 

This  brings  us  to  step  3 of  the  larger  instructional  model.  This  step  is  a variation  of  Dick,  Carey  and 
Carey’s  formative  evaluation  and  also  addresses  Jonassen’s  constructivist  model  in  which  teachers  adjust  task 
difficulty  or  redesign  the  task  to  accommodate  learners  who  are  experiencing  difficulties  in  completing  it.  The 
teacher  asks  questions  and  responds  to  student  answers  in  Step  3.1,  while  in  Steps  3.2,  3.3,  and  3.4,  the  teacher 
reverses  the  process,  soliciting  student  questions  and  responding  to  them  and  then  providing  feedback  on  students’ 
worksheet  responses.  The  goal  here  is  to  determine  if  the  instruction  was  effective,  to  identify  any  weaknesses,  and 
to  determine  where  and  how  instruction  should  be  revised  and  improved.  Formative  evaluation  occurs  while  the 
instruction  is  in  progress,  so  the  teacher  can  adjust  upcoming  instruction  to  meet  better  the  needs  of  students. 
Students  add  new  ideas  to  their  individual  concept  maps  at  various  times  during  the  unit  in  Step  3.5,  while  in  Step 
3.6  the  teacher  considers  how  best  to  improve  classroom  instruction  and  student  performance. 

Lastly,  Step  4 is  summative  evaluation  (Dick,  Cary  & Carey,  2005).  In  Steps  4.1  and  4.2,  the  teacher 
examines  the  culminating  activity  and  concept  maps  to  identify  how  well  students  learned  in  the  unit.  In  Steps  4.3 
and  4.4,  learners  complete  content  knowledge  and  attitude  and  behaviors  posttests.  The  teacher  considers  student 
responses  to  these  assessments  as  part  of  evaluating  the  effectiveness  of  the  unit  and  deciding  what  revisions  should 
be  made  for  future. 


Meta-Design  Principles 

We  also  derived  five  meta-principles  that  apply  to  the  design  of  the  unit  as  a whole.  These  meta-principles 
tell  us  when  to  apply  a rule  and  how  to  apply  it. 

Meta-principle  1:  Use  multiple  ways  of  learning  to  address  learner  differences. 

The  brain  processes  information  in  different  ways;  learners  may  see  different  connections  between  concepts 
if  information  is  presented  in  multiple  ways.  Gardner  (1993)  proposed  seven  intelligences  initially  and  argued  that 
everyone  is  born  with  potential  in  all  seven.  He  contended,  however,  that  cultural  and  personal  contacts  determine 
which  intelligences  develop.  The  seven  intelligences  include  linguistic,  logical-mathematical,  bodily-kinesthetic, 
spatial,  musical,  interpersonal,  and  intrapersonal.  Gardner  later  added  naturalistic  intelligence.  We  applied  this  meta- 
principle by  conveying  the  core  concepts  of  the  unit  using  as  many  modalities  as  possible  to  reach  the  various 
students.  Examples  include  having  students  investigate  optimal  areas  for  building  different  energy-generating 
facilities  (logical-mathematical  intelligence);  having  students  read  about  different  energy  sources  (linguistic 
intelligence);  having  students  manipulate  GIS  data  and  recognize  patterns  (spatial  intelligence);  and  having  students 
collaborate  on  group  activities  (interpersonal  intelligence).  We  used  different  entry  points,  such  as  telling  a story 
(linguistic  intelligence);  asking  questions  (logical-mathematical  intelligence),  displaying  real  objects  in  class 
(bodily-kinesthetic  intelligence),  and  doing  hands-on  activities  (bodily-kinesthetic  intelligence)  to  help  engage 
learners  with  the  topic. 
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Meta-principle  2:  Facilitate  the  process  of  modifying  instructional  materials  to  meet  the  needs  of  different  learners. 

According  to  Kinnaman  (1993),  teachers  play  a major  role  in  effective  implementations  because  they  can 
identify  how  to  modify  materials  to  address  the  needs  of  the  students  in  their  classes.  Cates  and  Kulo  (2009)  argued 
that  designers  should  provide  adequate  scaffolds  for  teachers  to  help  them  become  more  effective  in  supporting  their 
students.  In  this  way,  a good  design  for  teaching  and  learning  is  also  a professional  development  program.  We 
developed  two  sets  of  handouts,  a simplified  one  for  the  students  that  includes  scaffolds  for  the  task,  and  a detailed 
one  for  the  teacher  that  not  only  includes  scaffolds  for  the  task  but  also  scaffolds  for  what  students  are  supposed  to 
learn  and  the  thought  processes  involved.  We  provided  handouts  in  both  PDF  and  Word  format.  The  teacher  can 
choose  to  modify  the  activity  and/or  materials  based  on  how  learners  perform  the  task,  including  adjusting  task 
difficulty.  The  teacher  can  also  enrich  the  student  handout  by  adding  detailed  explanations  of  the  activity  copied 
from  the  teacher’s  handout. 

Meta-principle  3:  Use  icons  consistently  to  enhance  and  reinforce  student  learning  and  use  illustrations  to  reduce 
learner  dependence  on  text. 

According  to  Paivio  (1971)  images  act  as  mediators  in  learning  and  memory  tasks  and  can  serve  as  highly 
effective  memory  aids.  To  take  advantage  of  the  power  of  images,  we  used  icons  throughout,  in  materials  for  both 
teachers  and  students  and  we  illustrated  all  materials  extensively.  Our  use  of  icons  and  images  was  not  simply  to 
make  the  materials  more  attractive  but  also  clearer.  The  image  on  the  left  in  Figure  1 shows  the  student  resources 
Web  page.  Since  learners  use  this  page  almost  every  day,  we  used  images  that  depict  each  energy  source  to  facilitate 
navigation  and  also  to  enhance  and  reinforce  student  learning.  For  example,  we  used  an  image  of  a volcano  for 
geothermal  energy  and  an  image  of  a water  wheel  with  flowing  water  for  hydroelectric  energy.  We  also  used 
illustrations  with  arrows  to  connect  the  text  with  the  task  and  to  illustrate  the  task  (see  image  on  the  right). 

Figure  1.  Examples  showing  the  use  of  icons  and  illustrations 
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Meta-principle  4:  Use  procedural  facilitators  to  guide  learners’  responses. 

Scardamalia  and  Bereiter  (1986)  defined  procedural  facilitators  as  questions,  prompts,  and  simple  outlines 
of  important  learning  elements  that  teachers  use  to  scaffold  learning.  According  to  Rosenshine  and  Meister  (1992), 
such  scaffolds  provide  the  support  students  need  to  do  higher-level  thinking.  Tasks  in  the  energy  unit  have  steps  that 
must  be  done  in  sequence  to  get  the  desired  results.  Our  procedural  facilitators  are  designed  to  increase  learner 
comprehension  and  ensure  students  do  not  skip  steps.  For  example,  the  image  on  the  left  in  Figure  2 outlines  the 
steps  for  doing  a task,  while  the  image  on  the  right  shows  results  students  should  get  when  they  follow  those  steps. 
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Figure  2.  Illustrations  of  the  use  of  procedural  facilitators 


Meta-principle  5:  Use  Contrast,  Repetition,  Alignment,  and  Proximity  (CRAP)  design. 

According  to  Williams  and  Tollett  (2000)  there  are  four  basic  design  principles  one  should  apply  in 
designing  both  print  materials  and  Web  pages.  These  are  contrast,  repetition,  alignment,  and  proximity.  Figure  3 
illustrates  how  we  applied  these  principles.  We  employed  contrast  in  the  form  of  larger  san  serif  font  in  bold  to 
emphasize  key  words  and  by  using  a background  color  that  contrasts  well  with  the  text  and  images.  We  employed 
repetition  by  using  the  same  shades  of  green  and  white  colors,  consistent  fonts,  the  Environmental  Literacy  and 
Inquiry  (ELI)  logo  at  the  top,  and  light  bulb  navigation  buttons.  We  used  a larger  san  serif  font  for  the  rest  of  the 
body  text  and  used  left  alignment  for  items  on  the  Web  pages.  We  employed  proximity  by  placing  headings  and 
subheadings  closer  to  their  related  text  or  graphic  than  to  the  text  or  graphics  above  them.  The  Web  pages  also  have 
lots  of  white  space  to  help  reduce  cognitive  load  on  the  learners.  Our  intent  throughout  was  to  enhance  readability. 

Figure  3.  Illustration  of  CRAP  design 
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A Design  Case: 

Creating  an  Enhanced  Version  of  the  Diffusion  Simulation  Game 

Miguel  Lara,  Rodney  Myers,  Theodore  Frick,  Sinem  Aslan,  and  Theodora  Michaelidou 

Department  of  Instructional  Systems  Technology,  School  of  Education,  Indiana 

University  Bloomington 

In  this  paper,  we  describe  the  design  case  for  creating  a new  online  version  of  the  Diffusion  Simulation 
Game  (DSG).  Originally  a board  game  (Molenda  & Rice,  1979)  and  then  an  online  version  (Frick,  Ludwig,  Kim,  & 
Huang,  2003),  the  new  DSG  was  rapidly  prototyped  first  on  paper  and  then  in  Flex  Builder,  ActionScript,  PHP  and 
MySQL.  We  also  report  results  of  play-tests,  usability  evaluations,  and  how  the  design  and  implementation  of  the 
DSG  has  been  improved. 


Overview 

The  purpose  of  the  Diffusion  Simulation  Game  (DSG)  is  to  learn  change  management  strategies  which  are 
consistent  with  Roger's  (1995,  2003)  diffusion  of  innovation  theory  and  related  research  in  this  area.  The  goal  of 
the  DSG  is  to  promote  acquisition  of  strategies  that  result  in  adoption  of  an  innovation  (peer  tutoring)  by  the 
principal,  teachers  and  support  staff  at  a fictional  junior  high  school.  The  DSG  models:  progressive  stages  of 
awareness,  interest,  appraisal  and  trial  of  the  innovation  that  precede  adoption  by  individuals;  people  who  are 
innovators,  early  adopters,  early  majority,  late  majority,  and  laggards;  and  those  who  are  opinion  leaders  and 
gatekeepers.  Game  activities  include  gathering  information  on  potential  adopters;  viewing  diagrams  of  networks 
portraying  school  committees,  lunch  mates,  and  socializing  outside  of  school;  talking  to  individual  staff  members; 
asking  help;  making  site  visits;  providing  a demonstration  of  the  innovation;  conducting  workshops;  use  of  mass 
media;  confrontation;  and  compulsion. 

The  challenge  to  the  DSG  player,  who  takes  the  role  of  a change  agent,  is  to  get  as  many  individuals  as 
possible  to  adopt  the  innovation.  Each  diffusion  activity  takes  from  1 to  5 weeks  of  virtual  time,  and  the  game  ends 
when  either  72  weeks  elapse  or  all  teachers  and  the  principal  become  adopters.  By  repeated  play  of  the  DSG,  a user 
is  expected  to  learn  which  diffusion  strategies  are  effective  with  whom  and  when,  depending  on  the  adopter  type  and 
each  person's  changing  stages  of  adoption  when  appropriate  strategies  are  chosen  that  work  with  that  person.  It 
takes  about  two  hours  to  play  the  DSG  the  first  time,  and  about  an  hour  for  each  successive  play.  The  DSG  also 
models  stochastically  the  probabilities  of  success  for  various  diffusion  activities.  Sometimes  an  activity  succeeds 
and  other  times  it  does  not,  depending  on  the  current  game  state,  probabilities  of  success  of  the  activity  for  various 
adopter  types  and  stages  of  adoption,  and  chance.  If  used  as  part  of  a course — after  students  have  played  the  DSG 
multiple  times  outside  of  class — the  DSG  includes  a printed  summary  of  key  ideas  from  Roger’s  theory  and  change 
management  for  discussion  during  a debriefing  session. 

The  DSG  has  been  successfully  used  in  a board  game  format  for  several  decades  in  the  Instructional 
Systems  Technology  master's  program  at  Indiana  University  Bloomington.  In  2002,  the  third  author  led  a team  of 
graduate  students  to  build  the  first  online  version  of  the  DSG,  which  has  been  used  regularly  in  the  distance  master's 
program  (cf.  Frick  et  al.,  2003). 

Requests  for  DSG  licenses  for  use  outside  of  our  university  have  been  growing.  Starting  in  the  fall  2008, 
we  have  been  designing  and  developing  an  enhanced  version  of  the  DSG  that  will:  1)  store  logs  for  successive  game 
plays,  reviewable  by  students  and  their  instructors;  2)  make  it  easy  to  insert  new  content  for  different  settings  (e.g., 
in  business,  other  languages);  3)  make  it  easier  to  maintain,  grant  licenses  and  collect  license  fees;  4)  link  game  play 
to  a new  record  keeping  system  to  facilitate  research  on  strategies  players  use  and  how  well  they  leant;  5)  add  levels 
of  difficulty  to  the  game;  and  6)  improve  the  interface  to  increase  ease  of  use.  In  addition  to  applying  van 
Merrienboer's  (2007)  4C/ID  model  for  design,  we  are  creating  the  new  DSG  in  Flex  Builder,  ActionScript  and 
MySQL  so  that  it  will  run  over  the  Web  but  feel  like  a desktop  application. 

The  Importance  of  Diffusion  of  Innovation  Theory 

Innovation  is  the  process  of  transforming  an  opportunity  into  new  ideas  with  the  goal  to  improve  existing 
products,  practices,  or  services  (dal  Zotto  & Van  Kranenburg,  2008).  Notwithstanding  the  overall  positive  impact 
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that  an  innovation  might  have  within  a specific  context,  there  will  invariably  be  people  reluctant  to  adopt  it 
(Burkman,  1987).  This  is  mainly  due  to  the  fact  that  most  innovations  do  not  diffuse  by  themselves  but  require 
change  agents  and  the  willingness  of  potential  users  to  change  their  preexisting  mental  models  and  behaviors. 

“Diffusion  is  the  process  by  which  an  innovation  is  communicated  through  certain  channels  over  time 
among  the  members  of  a social  system”  (Rogers,  2003,  p.ll).  Diffusion  of  innovation  theory  explains  the  process  by 
which  new  ideas  and  practices  spread  between  and  within  social  systems  (Valente  & Davis,  1999).  Nowadays, 
multiple  disciplines  offer  courses  related  to  change  management  and  diffusion  of  innovations  as  part  of  their 
curriculum  to  provide  students  the  knowledge  and  skills  needed  when  dealing  with  the  implementation  of  new 
concepts  and  practices  as  part  of  their  profession. 


Purpose 

The  purpose  of  this  paper  is  to  describe  the  process  of  designing  a new  online  version  of  the  DSG  to 
overcome  several  limitations  of  the  previous  version;  among  the  new  version  improvements  are:  storing  interaction 
sequences  in  order  to  assess  improvement  in  gaining  adopters  across  repeated  play;  storing  game  logs  reviewable  by 
students  and  their  instructors;  making  it  easier  to  insert  new  content  for  different  settings  (e.g.,  in  business,  other 
languages);  making  it  easier  to  maintain,  grant  licenses  and  collect  license  fees;  adding  levels  of  difficulty  to  the 
game;  and  improving  the  interface  to  increase  ease  and  speed  of  use. 

Methods 

Participants 

A team  of  five  graduate  students  in  the  Instructional  Systems  Technology  (1ST)  graduate  program  at  a large 
midwestern  university  led  by  the  third  author  collaborated  in  the  design  and  development  process  of  the  DSG 
enhanced  version.  The  third  author  had  previously  led  another  research  group  in  the  design  and  development  of  the 
first  DSG  online  version  (Frick  et  al.,  2003). 

Design  Procedure 

Weekly  meetings  were  held  during  a semester.  Besides  the  meetings,  the  team  members  used  emails  and  a 
listserv  for  communication.  A Google  site  was  eventually  created  to  keep  track  of  all  of  the  decisions  made  during 
the  design  process. 

The  first  meetings  were  entirely  devoted  to  becoming  familiar  with  the  previous  DSG  versions.  One 
session  consisted  of  playing  the  board  version;  this  session  was  led  by  Dr.  Molenda,  one  of  the  original  creators  of 
the  game,  who  also  answered  questions  about  the  original  design. 

In  subsequent  meetings,  the  team  members  use  brainstorming  to  identify  a list  of  limitations  of  the  previous 
online  version  and  desired  enhancements.  This  list  was  then  used  to  define  goals  for  the  new  enhanced  DSG  version. 
The  goals  were  classified  in  two  different  categories: 

1)  Front-end  (user  interface): 

• Improve  the  interface  to  increase  ease  of  use,  reducing  the  amount  of  scrolling  and  mouse  clicks 
needed  to  play  the  game  and  eliminating  the  need  to  refresh  the  page. 

• Improve  the  feel  and  look,  making  it  more  visually  appealing. 

• Add  licensing  interface. 

2)  Back-end 

• Store  logs  for  successive  game  plays,  reviewable  by  students  and  their  instructors. 

• Make  it  easy  to  insert  new  content  for  different  settings  (e.g.,  in  business,  other 

• languages). 

• Link  game  play  to  a new  record  keeping  system  to  facilitate  research  on  strategies  players  use  and 
how  well  they  learn. 

• Add  levels  of  difficulty  to  the  game. 

Once  the  expectations  were  identified,  in  the  subsequent  sessions  the  team  dealt  with  two  issues 
simultaneously,  each  of  them  taking  approximately  half  of  each  session:  the  software  to  be  used  for  the  entire  project 
and  the  user  interface  (UI)  prototype. 
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Regarding  the  software  to  be  used,  the  team  decided  that  the  front-end  would  consist  of  SWF  files  created 
in  the  Adobe  open  source  program  called  Flex  3;  for  the  back-end  it  was  decided  to  use  a combination  of  PHP  (a 
server-based  scripting  language)  and  MySQL  (a  relational  database  management  system). 

The  reason  for  using  Flex  3 was  that  most  Web  browsers  nowadays  support  Adobe  Flash  player  which  can 
play  SWF  movies.  An  alternate  solution  was  to  use  AJAX  (asynchronous  JavaScript  and  XML),  a group  of 
technologies  that  can  be  used  to  create  Web  applications  that  do  not  require  the  browser  to  refresh  the  page; 
however,  the  team  realized  that  there  are  some  issues  related  to  browser  and  platform  compatibility  with  AJAX. 
Moreover,  the  learning  curve  to  configure  and  start  using  Flex  was  perceived  to  be  smaller  than  AJAX,  allowing  a 
faster  pace  in  the  development  of  the  application. 

Concerning  the  UI  design,  several  paper  and  digital  prototypes  were  first  created.  Based  on  those 
prototypes,  the  team  decided  to  use  the  concept  of  an  “Activity  Area”  (Figure  1)  in  which  the  player  would  drag  and 
drop  a specific  diffusion  activity  along  with  the  staff  members  selected.  Feedback  about  the  player’s  actions  would 
also  be  displayed  in  this  area.  This  prototype  helped  us  to  visualize  all  the  different  sections  needed  in  the  game  and 
assisted  us  in  determining  the  best  way  to  integrate  all  of  them.  The  team  quickly  decided  that  a drag-and-drop 
interface  was  less  efficient  than  simply  clicking  desired  components  to  select  them. 

Comparing  the  paper  prototype  to  the  first  online  version  of  the  DSG  (Figure  2)  it  can  be  noticed  that  the 
prototype  followed  a similar  layout,  using  three  different  tabs  to  contain  each  of  the  three  different  main  sections  of 
the  game,  which  are: 

1)  Game  Rules,  which  includes  the  directions  and  objectives  of  the  game; 

2)  Play  Game,  which  includes  the  simulation  game  itself; 

3)  Game  Logs,  which  includes  the  list  of  diffusion  activities  and  staff  members  selected  by  the  player 
during  each  turn. 

The  paper  prototype  (Figure  1)  shows  the  content  of  the  “Play  Game”  tab.  This  tab  also  kept  a similar 
layout  as  the  first  online  version:  the  calendar  section  remained  at  the  top  and  the  staff  member  list  on  the  left,  while 
the  activity  list  was  moved  to  the  bottom.  The  main  difference  between  the  prototypes  and  the  DSG  first  online 
version  was  the  introduction  of  the  “Activity  Area”  section. 
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Figure  1.  Example  of  DSG  paper  prototype. 
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ADOPTERS: 


Lunchmates  Diagram 

Staff  Members 

Social  Diagram 

STAFF 

MEMBER 

INFORMATION 

AWARE- 

NESS 

INTEREST 

TRIAL/ 

APPRAISAL 

ADOP- 

TION 

A 

Principal 

Very  ambitious  (has  a 20-year  plan);  member  of  the  Rotary  Club  and  local 
RepubHcan  Club  (active  in  both);  delegates  authority  to  able  administrative 
assistants  and  runs  a "tight  ship.”  Has  a "master's-plus’  in  administration. 

□ 

B 

Secretary 

Has  been  in  this  school  smce  it  was  built  and  quite  indispensable  to  its 
smooth  functioning.  Runs  most  faculty  social  functions. 

1 

1 

C 

Janitor 

Fond  of  children,  but  stern.  He  tends  to  allow  extensive  use  of  the  school 
building,  but  has  strict  rules  and  is  inflexible  about  infractions. 

1 

II 

D 

Math  Chairman 

A veteran  in  the  school,  he  runs  the  most  experienced  department  with  a 
minimum  of  effort  Is  involved  much  more  in  out-of-school  activities  such  as 
the  local  garden  club  and  conservation  organization.  Still  regrets  the  repeal  of 

□ 

E 

Math  Teacher 

Working  hard  to  complete  her  Master’s  thesis  because  the  salary  increment 
will  supplement  husband's  uncertain  income  from  new  construction  business. 
Acts  as  attendance  coordinator  and  has  tended  to  get  somewhat  sour  about 
students  and  their  parents. 

□ 

F 

Math  Teacher 

□ 

G 

Science  Chairman 

□ 

H 

Science  Teacher 

□ 

STAFF 

MEMBER 

INFORMATION 

AWARE- 

NESS 

INTEREST 

TRIAL/ 

APPRAISAL 

ADOP- 

TION 

Information 

COST 

INFORMATION 

1 

Personal:  Find  out  what  sort  of  oeoDle  the  staff  members  are. 

week 

1 

Lunchmates:  Observe  carefully  to  see  who  lunches  with  whom 

week 

each  noon. 

1 

Committees:  Find  out  who  are  members  of  the  various  formal 

week 

committees  set  up  in  the  school. 

2 

Social:  Observe  the  out-of-school  social  patterns  to  leam  who  plays 

weeks 

poker  together,  who  bowls  together,  etc. 

Diffusion  Activities 

COST 

DIFFUSION  ACTIVITY 

1 

Talk  To:  You  make  a conscious  effort,  over  a period  of  about  one 

week 

week,  to  engage  any  ONE  person  in  a number  of  one  to  one 
conversations. 

1 

Ask  HelD:  You  ask  anv  ONE  of  the  staff  for  advice  or  for  helo  in  one 

week 

of  your  projects...  preparing  some  learning  materials,  setting  up  a 
demonstration,  running  a workshop,  etc. 

2 

Pilot  Test:  You  attempt  to  influence  ONE  teacher  by  asking  to  let 

weeks 

you  conduct  an  informal  pilot  test  of  peer  tutoring  with  his/her 
students. 

4 

Site  Visit:  You  select  any  FIVE  persons  to  visit  Lighthouse  School, 

weeks 

in  the  next  state,  where  an  exemplary  tutoring  program  is  in 

Droaress. 

Figure  2.  DSG  first  online  version. 


Figure  3.  First  digital  prototype. 

Corry,  Frick  and  Hansen  (1997)  highlight  the  importance  of  focusing  on  the  user  throughout  the  design  and 
development  processes.  An  essential  objective  in  the  design  of  the  enhanced  online  version  of  the  DSG  was  to 
provide  a better  user  experience.  After  a deeper  analysis  of  the  paper  prototype,  the  team  realized  that  placing  the 
diffusion  activities  near  the  bottom  of  the  screen  created  the  potential  disadvantage  of  players  having  to  scroll  down 
every  time  they  needed  to  select  an  activity.  For  this  reason,  the  team  decided  to  have  all  the  activities  listed  one 
below  the  other,  on  the  right  side  of  the  activity  area. 
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At  this  point,  the  team  had  already  made  the  decision  to  use  Flex  for  the  User  Interface.  Since  Flex 
facilitates  rapid  prototyping  (Tripp  & Bichelmeyer,  1990),  the  team  decided  to  use  it  to  create  the  first  digital 
prototype  that  would  include  the  modifications  suggested  from  the  paper  prototype  (Figure  3). 

The  first  digital  prototype  helped  the  team  to  have  a clearer  idea  about  the  monitor  resolution  that  would  be 
needed  to  fit  as  many  of  the  game  elements  as  possible  in  a single  screen.  A monitor  resolution  of  1024  x 768  was 
the  most  appropriate  to  diminish  the  amount  of  vertical  scrolling  while  preventing  horizontal  scrolling  at  the  same 
time. 

Even  though  the  first  digital  prototype  was  not  fully  functional,  designers  were  able  to  emulate  playing  the 
game  by  selecting  diffusion  activities  and  staff  members.  Through  this  interaction,  designers  noticed  that  players 
would  need  to  move  the  mouse  from  one  side  of  the  monitor  to  the  other  for  every  single  turn  in  the  game.  Designers 
also  realized  that  the  natural  order  of  the  “Activity”  and  “Staff  members”  sections  were  inverted  because  players 
need  to  first  select  an  Information  or  Diffusion  Activity  and  then  select  up  to  five  staff  members. 

Based  on  these  observations,  the  designers  switched  these  sections  and  placed  them  next  to  each  other  to 
match  the  sequence  of  decision  making.  Furthermore,  since  players  of  the  first  online  version  would  continue 
playing  in  the  new  version,  the  designers  decided  to  keep  a similar  background  color  as  the  first  version.  All  these 
changes  are  shown  in  Figure  4. 

The  UI  design  process  was  very  iterative;  as  soon  as  a change  was  done,  all  designers  accessed  the 
prototype  and  analyzed  the  revisions  (even  though  the  game  was  not  functional  yet).  During  team  meetings,  the 
designers  commented  about  their  observations  and  new  changes  were  decided  based  on  a team  consensus. 

After  several  weeks  of  work  on  the  UI,  the  team  agreed  upon  the  design  shown  in  Figure  5.  In  this  final 
design  the  diffusion  activity  section  is  located  at  the  left,  followed  by  the  staff  member  section,  following  the  natural 
left-to-right  reading  process  (as  mentioned  above,  players  need  to  first  select  a diffusion  activity  and  then  the  desired 
staff  members). 


TKe-  V uffuticm/  S C*nula£ti>n/  (Jcttne- 


Game  Rules  Play  Game  Game  Log 


Figure  4.  Second  digital  prototype. 


The  designers  recognized  the  importance  of  reducing  the  short-term  memory  load  as  a golden  rule  of 
interface  design  (Shneiderman  & Plaisant,  2005).  However,  this  golden  rule  defied  the  rule  about  minimizing  the 
amount  of  scrolling  as  much  as  possible.  For  purposes  of  the  game,  a player  has  to  get  the  personal  information  of 
each  staff  member.  Displaying  all  this  information  all  of  the  time  results  in  excessive  vertical  scrolling.  The  solution 
we  proposed  was  to  have  two  views:  1)  a list  view  showing  only  staff  member  job  titles  along  with  a “tooltip”  (call 
out)  containing  each  staff  member’s  description  whenever  the  mouse  hovers  over  the  job  title;  and  2)  a detailed 
view,  that  shows  each  staff  member’s  complete  description  (only  if  “Get  Personal  Info”  is  true  for  him  or  her). 
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Figure  5.  Final  digital  prototype:  List  View. 


Figure  6.  Final  digital  prototype:  Detailed  View 


Moreover,  the  designers  also  decided  to  add  more  graphics,  including  the  face  of  each  of  the  staff  members 
for  the  players  to  associate  with  a person’s  title,  to  make  the  game  more  appealing.  The  first  set  of  faces  was  created 
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using  an  online  commercial  product  called  SitePal  which  specializes  in  the  design  of  virtual  avatars.  However,  for 
the  release  version  of  the  new  online  game,  the  designers  have  decided  to  use  cartoon  (posterized)  versions  of 
pictures  from  real  faculty,  staff  and  students.  The  faces  of  the  staff  members  along  with  their  personal  description 
are  included  under  the  “Detailed  View”  (Figure  6). 

In  conjunction  with  the  UI  design  of  the  “Play  Game”  tab  contents,  a designer  started  working  on  the  design  of 
the  “Game  Rules”  tab.  Simultaneous  work  on  the  UI  of  both  tabs  was  possible  thanks  to  the  use  of  Flex  components, 
which  are  modular  and  easily  combined  into  the  final  application. 

In  the  first  online  version,  the  “Game  Rules”  tab  consists  of  a single  page.  Given  the  goal  of  reducing  scrolling, 
the  designers  decided  to  divide  the  content  into  five  different  subsections,  each  being  displayed  after  clicking  on  the 
corresponding  button  on  the  left  side  of  the  screen  (Figure  7). 


Figure  7.  Final  digital  prototype:  Game  Rules  tab 


Back-end  Development 

Once  the  UI  design  was  completed,  the  designers  created  the  database  structure  using  an  Entity- 
Relationship  approach  that  would  allow  to:  1)  store  logs  for  successive  game  plays  2)  make  it  easy  to  insert  new 
content  for  different  settings  and  languages  3)  make  it  easier  to  maintain,  grant  licenses  and  collect  license  fees;  and 
4)  add  levels  of  difficulty  to  the  game.  The  database  used  was  the  open-source  DBMS  MySQL.  It  was  chosen 
mainly  because  most  designers  were  already  familiar  with  it  and  they  were  certain  that  it  was  going  to  handle  the 
data  to  be  stored  without  any  issues.  Furthermore,  this  database  was  already  being  used  for  other  projects  within  the 
department  and  there  was  no  need  to  install  a new  database  server. 

The  designers  used  the  Model-View-Controller  architecture  for  the  development  of  the  DSG.  The  model 
consisted  of  model  objects  using  PHP,  the  view  consisted  of  the  Flex  user  interface,  and  the  controller  consisted  of 
the  ActionScript  programs  and  their  corresponding  calls  to  PHP  programs  which  ultimately  communicated  with  the 
database  to  retrieve  and  store  data. 

In  order  to  be  able  to  determine  if  learning  is  promoted  by  playing  the  game,  the  designers  analyzed  the 
player-game  interactions  that  needed  to  be  collected  as  evidence  for  assessment.  This  process  of  identifying  relevant 
interactions  and  creating  mechanisms  to  collect  and  store  data  is  similar  to  the  “information  trail”  suggested  by  Loh 
(2008).  The  designers  then  plan  to  use  MAPSAT  methods  (Frick,  Myers,  Thompson,  & York.,  2008)  to  analyze  the 
data  gathered  to  determine  the  presence  of  patterns  in  the  strategies  used  that  result  in  either  gaining  many  adopters 
(expert  play)  or  few  adopters  (novice  play). 
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Usability  Testing 


Testers.  The  designers  used  convenience  sampling  to  recruit  the  testers  for  the  usability  testing.  A total  of 
seven  persons  were  recruited:  six  of  them  were  female  and  one  male.  Three  of  the  females  were  doctoral  students  in 
different  areas  in  the  School  of  Education.  Most  of  the  participants  were  Americans  except  for  two  who  were  from 
Turkey. 

Requirements  in  recruiting  testers  were: 

1)  They  should  not  have  played  any  version  of  the  DSG  before. 

2)  They  should  not  be  familiar  with  diffusion  of  innovation  theories. 

The  usability  tests  were  conducted  on  an  individual  basis,  with  each  designer  taking  the  role  of  observer 
while  a tester  was  playing  the  game.  Testers  were  encouraged  to  use  their  own  computers  or  laptops  to  avoid  any 
disadvantages  resulting  from  using  unfamiliar  technology  (one-button  mouse,  operating  system,  etc.).  They  were 
informed  that  the  usability  test  could  take  up  to  two  hours  and  that  they  could  take  as  many  breaks  as  they  needed 
during  that  time.  They  were  also  asked  to  think  aloud  (Preece  et  al.,  1994)  while  playing  the  game.  They  were  not 
helped  during  the  game  unless  it  was  absolutely  necessary. 

There  were  no  specific  questions  or  tasks  that  testers  had  to  complete  during  the  usability  test.  They  were 
only  asked  to  play  the  game  and  to  think  aloud  as  they  interacted  with  the  game.  The  observer  took  notes  and 
occasionally  prompted  the  tester  to  think  aloud  while  playing. 

After  completing  the  game,  the  testers  were  asked  the  following  questions: 

• Was  it  difficult  for  you  to  play  the  game? 

• Was  it  difficult  for  you  to  navigate  in  the  game? 

• What  would  you  do  differently  if  you  were  to  play  the  game  again? 

• Is  there  anything  you  would  suggest  to  improve  your  game  experience? 

Usability  test  results.  After  completing  all  usability  tests,  the  designers  met  and  discussed  their  findings. 
Each  designer  described  the  issues  or  problems  that  the  testers  had  as  they  were  playing  the  game.  For  each  issue  or 
problem  identified,  the  rest  of  the  designers  confirmed  if  they  had  experienced  something  similar.  A list  of  the  issues 
along  with  the  number  of  times  it  was  observed  across  multiple  tests  was  created.  The  list  of  issues  was  then 
grouped  into  two  categories:  game  mechanics  and  user  interface. 

Game  mechanics: 

• Testers  needed  to  invest  five  to  ten  minutes  to  understand  the  rules  of  the  game. 

• Testers  were  uncertain  about  which  staff  members  had  become  adopters. 

• Testers  assumed  that  they  would  be  allowed  to  select  personal  information  for  only  five  staff  members 
during  the  entire  game  instead  of  during  a single  turn. 

User  interface: 

• Some  game  sections  were  unnoticed  by  the  testers  such  as  “Detailed  View”  and  “Game  Logs.” 

• Testers  were  not  always  sure  which  staff  received  points  (did  not  see  red  boxes  get  filled  when  points 
were  awarded  after  reading  feedback). 

• Testers  wondered  if  it  cost  one  week  each  time  they  clicked  on  the  link  to  see  a diagram. 

• Feedback  contains  letters  to  refer  to  staff  members,  but  users  were  not  sure  what  their  role  is  without 
looking  to  the  left  and  possibly  scrolling. 

• Some  testers  did  not  know  what  a “Home  Ec  Teacher”  is. 

• Testers  wanted  to  display  more  than  one  diagram  at  a time  but  could  not. 

Regarding  the  game  mechanics  issues  identified,  it  is  worth  mentioning  that  usability  testers  were  not  given 
any  information  about  the  diffusion  of  innovation  theory  before  playing  the  game.  This  might  be  unrealistic  in  real 
game  play  because  some  of  the  players  who  will  access  the  game  most  likely  would  have  already  studied  the  theory. 
Nevertheless,  these  findings  clearly  expose  a strong  weakness  in  the  game:  it  takes  too  long  for  players  to 
understand  the  directions  and  objectives  of  the  game.  Potential  players  could  be  easily  discouraged  from  playing  the 
game  if  this  issue  is  not  addressed.  Alternative  solutions  consist  of  revising  the  Game  Rules  section  to  provide  a 
clearer  and  more  streamlined  version  of  the  directions;  adding  just-in-time  information  in  the  game  interface; 
providing  hints  on  an  as-needed  basis,  based  on  the  player’s  interaction;  creating  a concise  video  tutorial. 

The  user  interface  issues  detected  made  the  designers  realize  the  usefulness  of  usability  testing  as  an 
ongoing  part  of  the  design  process.  Addressing  the  issues  detected  by  the  testers  will  help  to  improve  the  game’s 
usability. 
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Future  Work 

So  far,  the  designers  have  completed  the  front-end  and  back-end  design  of  the  DSG  v.  2.0.  The  usability 
testing  conducted  was  mainly  to  test  the  front-end — i.e.,  the  interactions  of  the  players  with  the  UI.  The  back-end 
stores  information  about  all  of  these  interactions,  allowing  players  to  resume  their  games. 

The  next  task  is  to  recruit  more  testers  to  play  multiple  games  and  to  analyze  the  data  collected  using 
MAPS  AT  (Map  & Analyze  Patterns  & Structures  Across  Time)  to  verify  if  learning  about  the  implementation  of 
effective  diffusion  strategies  is  actually  occurring.  Version  2 of  the  DSG  will  allow  us  to  conduct  further  research 
about  student  learning  while  playing  the  game.  Measuring  whether  or  not  learning  is  occurring  through  playing 
educational  games  or  simulations  can  be  challenging  (Bredemeier  & Greenblatt,  1982;  Dempsey,  Rasmussen,  & 
Lucassen,  1996;  Hays,  2006;  cf.  Thiagarajan,  2003).  In  the  case  of  the  DSG,  if  the  total  number  of  adopters  obtained 
by  the  same  student  across  multiple  game  plays  has  increased,  we  are  assuming  that  some  kind  of  learning  has 
occurred.  Whether  this  learning  is  consistent  with  Roger’s  theory  of  diffusion  of  innovations  is  something  that  we 
plan  to  investigate.  We  intend  to  do  this  by  use  of  MAPSAT  to  study  patterns  of  diffusion  activities  chosen  by 
players  who  are  highly  successful  (“experts”  who  get  everyone  or  nearly  everyone  to  adopt)  compared  with  players 
who  only  get  some  of  the  adopters  (“non-experts”). 

Given  the  flexibility  in  the  new  back-end  design,  future  DSG  versions  will  include  different  levels  of 
difficulty  and  multiple  language  support;  we  are  also  exploring  the  possibility  of  creating  a version  using  a business 
context. 
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Time-Saving  Online  Course  Tools  and  Features  that  Promote  Self-Regulation, 
Minimize  Extraneous  Cognitive  Load,  and  Support  Learning  Community 


Kathryn  Ley 

University  of  Houston  - Clear  Lake 

This  paper  describes  content-independent  online  course  communication  tools  and  features  designed  to  minimize 
extraneous  cognitive  load  (ECL),  support  self-regulation  (SR)  and  promote  the  learning  community.  Online  course 
communication  tools  and  features  can  enable  an  instructor  to  help  learners  more  efficiently;  they  allow  learners  to 
spend  less  time  and  effort  locating,  organizing,  accessing,  distributing,  and  retrieving  course  information.  Job  aids 
and  procedure  templates  to  easily  incorporate  online  tools  and  features  designed  to  mitigate  ECL  and  facilitate  SR 
allow  instructors  into  their  existing  online  courses.  Furthermore,  online  features  and  tools  consistent  with  CL  and  SR 
design  principles  passively  promote  the  course  learning  community  in  small  but  significant  ways.  Example  job  aids 
and  templates  designed  and  developed  during  an  iterative,  formative  evaluation  of  instructor  training  (Ley,  2009) 
which  applied  an  instructional  development  methodology  (Richey  & Klein,  2007)  are  included  and  discussed.  The 
training  development  process  study  confirmed  that  faculty  online  communication  problems  may  be  addressed  by  job 
aids  and  procedure  templates  derived  from  CL  and  SR  theory.  Instructors  may  use  job  aids  and  templates  to 
structure  SR  support,  to  minimize  ECL,  and  to  promote  learning  community. 

Extraneous  Cognitive  Load,  Self-Regulation,  Learning  Communities  and  Learning 

SR,  ECL  and  learning  community  research  and  theory  have  important  implications  for  online  learning.  Research 
results  are  consistent  with  the  instructional  implications  of  the  corresponding  theory  and  associated  corroborating 
research.  The  three  phenomena  synergistically  interact  to  contribute  to  online  learning.  ECL  impedes  learning  and 
can  obstruct  learning  community  participation;  instructional  features  and  tools  that  support  SR  and  minimize  ECL 
also  may  contribute  to  learning  community  participation.  Three  Online  Learning  Influences  CL,  a learner’s  mental 
effort  and  cognitive  resources  expended  when  learning  (Paas,  Renkl,  & Sweller,  2004;  van  Merrienboer  & Ayres, 
2005)  is  inherently  greater  in  online  courses  than  in  on-ground  courses  because  the  online  interface  imposes  ECL 
not  required  in  face-to-face  (f2f)  instruction;  computer-mediated  communications  require  more  skill  and  effort  than 
do  f2f  interactions.  Of  the  three  CL  categories  associated  with  learning;  germane,  intrinsic,  or  ECL  (Gerjets, 

Scheiter,  & Catrambone,  2004);  ECL  is  most  problematic  for  instruction  and  especially  online  instruction.  Whereas 
germane  CL  engages  learners  in  creating  new  schemas  and  intrinsic  CL  is  a function  of  the  number  of  elements  a 
learner  must  understand  (Gerjets,  Scheiter,  & Catrambone),  only  ECL  undermines  rather  than  contributes  to  learning 
(DeLeeuw  & Mayer,  2008)  and  only  ECL  increases  with  less  than  optimal  design  (van  Gog,  Paas,  & van 
Merrienboer,  2004).  On  the  other  hand,  minimizing  ECL  associated  with  online  information  and  communications 
simultaneously  encourages  SR  and  learner  participation  in  the  online  learning  community. 

Learner  SR  may  be  defined  as  the  self-generated  thoughts,  feelings,  and  actions  for  attaining  educational  goals 
and  “includes. . .planning  and  managing  time;  attending  to  and  concentrating  on  instruction;  organizing,  rehearsing 
and  coding  information;  establishing  productive  work  environment  and  using  social  resources  effectively.  . . it 
involves  self-efficacy,  outcome  expectations,  task  interest  or  valuing,  a learning  goal  orientation,  and  self- 
satisfaction  with  one’s  learning  and  performance”  (Zimmerman,  2004,  p.  139-140).  SR  foments  academic  success 
and  differentiates  the  more  effective  learner  from  the  less  effective  one.  Instructional  SR  support  has  increased 
learner  SR  activities  (Butler,  1998)  and  improved  their  learning  (Lan,  1996).  Learners  can  become  self-regulated 
through  teaching  and  instructional  activities  (Osman  & Hannafin,  1992;  Zimmerman,  2002).  Several  studies  have 
“provided  concrete  evidence  that  [self] -regulation  strategies  may  be  embedded  within  instruction”  (Osman  & 
Hannafin,  1992,  p.  88)  and  “each  self-regulatory  process  or  belief  such  as  goal-setting,  strategy  use,  and  self- 
evaluation  can  be  learned  from  instruction  and  modeling”  (Zimmerman,  p.  69).  Thus  the  instructor  who  models  SR 
and  provides  SR  support  within  an  online  course  encourages  learning.  Minimizing  ECL  and  encouraging  SR 
contributes  to  efficient  online  communications,  an  essential  attribute  of  an  online  learning  community. 

Learner  communications,  the  cornerstone  of  an  online  learning  community,  are  the  quintessential  form  of  online 
participation.  Online  learning  communities  depend  upon  each  learner’s  ability  to  participate.  Two  conditions  affect 
an  online  learner’s  participation  ability,  his  or  her  web-based  self-efficacy  and  online  course  design.  Web-based  self- 
efficacy  “refers  to  a learner’s  beliefs  about  his  or  her  capabilities  in  using  the  functions  of  the  [virtual  learning 
community]  website”  (Chen,  Chen,  & Kinshuk,  2009,  p.  135).  Web-specific  self-efficacy  positively  and 
significantly  affected  400  college  students’  intentions  to  share  knowledge  within  a learning  community.  Therefore 
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“cultivating  learners’  capabilities  ...  in  a virtual  learning  community  can  help  them  not  only  learn  more  efficiently, 
but  also  leant  ‘how  to  learn'  . . . [and]  prerequisite  training  for  using  the  [learning  community]  website  functions  . . . 
is  also  required”  (Chen,  p.144).  Although  online  learning  communities  may  arise  spontaneously,  they  may  also  be 
shaped  by  design  (Ke  & Hoadley,  2009).  Qualitative  data  from  more  than  400  online  graduate  students  indicated 
“structural  dimensions  of  the  course  were  perceived  as  the  responsibility  of  a thoughtful  instructor  who  understood 
learning”  and  who  provided  a “clear  course  structure”  that  engaged  learners  and  supported  them  with  a coherent 
course  framework  (LaPointe  & Reisetter,  2008,  p.  649).  Design  that  promotes  online  learning  community  values 
will  maintain  consistency  and  predictability  (Snyder,  2009);  “the  learning  environment  should  be  consistent  and 
predictable  so  that  the  learner  can  focus  on  the  content  and  not  the  delivery  environment”  (p.  56).  ECL-reducing 
and  SR-supporting  online  tools  and  features  unavoidably  introduce  consistency  and  predictability  because  they 
codify  communication  and  information  exchange,  retrieval,  and  access. 

Barriers  to  Instructor  Online  Participation 

Online  course  features  and  tools  promoting  SR  and  minimizing  ECL  necessarily  save  time,  a special  benefit  to 
online  faculty.  Faculty  who  teach  online  and  those  who  demur  to  do  so  often  lament  the  time  demands  associated 
with  online  teaching  (Mupinga  & Maughan,  2008;  Rockwell,  Schauer,  Fritz,  & Marx,  1999;  Visser,  2000; 
Zuckweiler,  Schniederjans,  & Ball,  2004).  When  asked,  faculty  ranked  time  demands  as  the  top  online  teaching 
challenge  (Vaughan,  2007).  Some  online  communications  time  requirements  stem  from  unavoidable  sources. 

Human  physiology  explains  some  of  the  intuitively  justifiable  discrepancy  between  online  and  face-to-face  teaching 
time  requirements.  Audio  interactions  communicate  more  quickly  then  text  given  approximately  180  spoken  words 
per  minute  (WPM;  Colbert,  1988),  50  typed  WPM;  350-450  heard  WPM  versus  185-300  read  WPM  (Fulmer,  1976); 
therefore,  face-to-face  instructor-student  listening-speaking  interactions  take  about  half  to  one-third  as  much  time  as 
typical  online  reading-typing  interactions. 

Studies  investigating  online  learning  have  revealed  online  instruction  can  take  more  time  than  face-to-face 
instruction.  Online  communication  usually  requires  more  time  than  does  comparable  f2f  communication  although 
the  extra  time  grows  exponentially  when  instructors  and  learners  unwittingly  introduce  ECF.  Online  email 
communications  took  substantial  online  instructor  teaching  time  (Zuckweiler,  Schniederjans,  & Ball,  2004).  Online 
interactions  demanded  both  faculty  and  learner  time  although,  as  faculty  gained  online  experience,  they  adapted 
ways  to  organize  and  decrease  email.  Email  may  be  the  preferred  online  communications  method  for  private,  one-to- 
one,  instructor-learner  interactions  because  email  appropriately  preserves  privacy  for  assignment  feedback  and 
sensitive  discussions  concerning  the  student’s  grades  or  other  personal  circumstances.  On  the  other  hand  email 
perpetrates  information  exclusivity,  foments  redundant  communications,  and  impedes  information  retrieval 
especially  when  employed  for  instructor-student  communications  devoted  to  enabling  learners  to  understand, 
interpret,  comprehend,  and  complete  instructional  activities  and  assignments.  Email  used  to  disseminate  course 
instructional  communications  that  every  learner  should  have  unnecessarily  creates  undesirable  ECL.  On  the  other 
hand,  many  features,  tools,  and  procedures  that  decrease  learner  ECL  simultaneously  support  both  instructor  and 
learner  SR;  instructors  benefit  from  the  same  features  and  tools  that  help  their  learners. 

Course  Design  and  ECL,  SR,  and  Participation 

The  ways  in  which  instructors  and  learners  display  and  exchange  assignment  and  course  content  information 
either  support  or  undermine  learning  and  learning  communities.  Less  than  optimal  course  logistics  and  navigational 
design  become  a significant  source  of  ECL.  Features,  tools,  and  procedures  that  organize  information,  structure 
communications,  cue  instructional  relevance,  time  frames,  and  course  events  reduce  ECL,  support  SR,  and  facilitate 
participation  (Figure  1). 
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Online  Course  Design  & Learning 


Figure  1 . Online  course  design  and  learning 

SR  and  CL  instructional  principles  (Clark,  Nguyen,  & Sweller,  2006;  Ley  & Young,  2001)  offer  a framework  for 
online  course  design  that  facilitates  immediate,  timely  course  instruction  and  information  access,  retrieval,  and 
dissemination.  Instructors  using  such  a framework  can  replace  unnecessarily  redundant  or  private  communications 
with  more  efficient  and  effective  interactions.  These  SR  and  CL  course  features,  tools,  and  procedures  increase  the 
ratio  germane  and  intrinsic  CL  time  and  effort  to  ECL  time  and  effort;  in  other  words,  they  spend  more  time  on 
teaching  and  learning  and  less  time  on  locating,  confirming,  clarifying,  and  disseminating  instructional 
communications  and  information.  Communications  grounded  in  CL  and  SR  theory  reduces  online  teaching  and 
learning  effort  devoted  to  ECL  and  to  self-regulating  because  they  reduce  the  former  and  facilitate  the  latter. 


SR-Supporting,  ECL-Minimizing  Communication  Features,  Tools,  and  Procedures 


Three  dedicated  locations  can  organize  most  of  the  course  communications  about  course  content  or  completing 
assignments.  The  first  location,  a dedicated  instructor  message  discussion  board,  disseminates  and  organizes  a 
searchable  permanent  archive  of  instructor  communications  accessible  to  every  participant.  Students  and  the 
instructor  have  one  primary  place  to  find  important  instructor  communications  initiated  during  the  course.  A 
dedicated  board  promotes  equity  since  everyone  has  continual  and  simultaneous  access  to  all  the  instructor 
information.  An  instructor  can  maintain  the  organization  and  avoid  ECL  (Figure  2)  in  several  ways:  (a)  posting  all 
his  or  her  text  communications  on  the  board  and  avoiding  email  or  other  locations  for  important  text  instructions  for 
the  class,  (b)  removing  any  student  messages  posted  on  the  instructor  board,  and  (c)  including  a syllabus  policy 
about  the  procedures  for  the  first  two  items.  This  eliminates  most  ECL  incurred  when  students  have  several  places  to 
look  for  current  instructor  course  communications. 


Figure  2.  Instructor  board  messages  preferable  to  email. 

One  of  the  most  effective  ways  for  organizing  and  equitably  answering  student’s  assignment  or  content  questions 
is  to  dedicate  a single  message  board  for  all  written  student  questions  and  comments.  The  instructor  can  maintain 
organization  and  equitable  access  in  several  ways.  First,  reposting  every  student’s  email  question  that  violates  your 
syllabus  policy  in  a discussion  board  message  beginning  by  thanking  the  anonymous  asker  and  introducing  the 
question  with  a comment  such  as  “Here  is  a great  example  of  a question  that  should  be  post  on  the  dedicated 
students  questions  board  since  everyone  should  see  my  response.”  To  avoid  embarrassing  the  emailing  question 
poser,  an  instructor  would  be  wise  to  delete  any  student  identifier  in  the  email  message  before  publishing  the 
question  on  the  students’  questions  discussion  board  message  and  responding  to  the  private  email  by  telling  the 
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student,  he/she  will  find  the  answer  on  student  discussion  board  dedicated  to  their  questions.  To  encourage  course 
participation,  students  should  also  be  able  to  talk  with  the  instructor  by  telephone,  in  person,  or  other  synchronous 
audio  communication  tools;  codifying  the  procedure  in  the  syllabus  provides  predictability  and  structure  required  for 
building  a learning  community. 

A third  discussion  board  dedicated  to  students  helping  each  other  allows  them  to  ask  and  answer  each  other’s 
questions.  As  with  the  dedicated  instructor  board  and  the  student  questions  board,  establishing  the  procedure  for 
when  and  how  students  use  the  board  in  the  syllabus  provides  predictability  and  structure  required  for  building  a 
learning  community.  Three  dedicated  discussion  boards  organize  communications  from  the  students,  among  the 
students,  and  from  the  instructor  (Figure  3).  Dedicated  discussion  boards  organize  instructional  communications  and 
thus  support  instructor  and  student  SR;  the  instructor  has  all  student  text  questions  chronologically  listed  in  one 
location. 

Self-regulating  online  faculty  and  students  monitor  their  progress,  a critical  SR  process  effectively  supported 
online  with  two  easily-embedded  course  features,  an  assignment  table  (Figure  4)  and  course  home-page  or 
announcement  page  reminders  (Figure  5). 
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Figure  3.  Three  discussion  boards  organize  communications  from  the  students,  among  the  students,  and  from  the 
instructor. 


Figure  4.  Assignment  Table  with  due  dates, 
submission  location  codes  & legend,  points 
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Figure  5.  Assignment  Reminders  preceded  by 
month_due  date 


An  assignment  table  feature  displays  every  assignment  chronologically  by  due  date  in  one  location  accessible  in  the 
course  and  included  in  the  syllabus.  The  first  three  columns,  course  week,  assignment  due  month  and  date  precede 
the  assignment  name  column,  coded  submission  location  column,  and  total  possible  points  column.  A legend  below 
the  six-column  table  translates  the  fifth  column  submission  codes.  The  assignment  reminders  announce  the  next  few 
assignments  chronologically  by  due  date.  The  table  and  message  reminders  duplicate  at  least  the  next  few 
assignments  due  which  temporarily  violates  an  SR  principle  - avoid  duplicating  information  which  potentially 
increases  ECL  for  learners  who  take  time  to  verify  and  confirm  redundancy  or  seek  to  resolve  conflict.  On  the  other 
hand,  the  ephemeral  messages  conveniently  remind  online  students  what  is  next  and  the  instructor’s  progress 
evaluating  the  assignment  status  after  due  date,  such  as,  evaluation  pending,  points  posted,  feedback  available,  etc. 

Assignment  filenames,  another  organizing  feature,  label  files  with  student  and  assignment  idetnifiers.  A syllabus 
policy  or  assignment  instructions  which  require  students  to  name  any  assignment  files  beginning  with  the  student’s 
last  and  first  initial  or  last  name  and  first  initial  followed  by  the  month  underscore  due  date  organizes  files  (Figure 
6).  Intuitively  named  files  enable  an  instructor  to  quickly  locate  a specific  assignment  for  a specific  student  or  to 
order  all  assignments  with  one  click  by  the  student’s  last  name  as  listed  in  a grade  book  or  database.  The  instructor 
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who  downloads  and  names  assignments  posted  in  message  boards  can  use  the  same  convention;  for  an  entire  class’s 
collated  assignment,  the  filename  would  substitute  all  for  student  identifier,  e.g.,  a July  2 1 sl  assignment  = 7_2 1 all 
(Figure  6,  first  file  listed). 

The  online  features  described  in  this  article  work  in  conjunction  with  common  online  tools;  together  they 
facilitate  instructor  SR  and  student  SR  while  minimizing  ECL  and  promoting  learning  community.  The  features 
described  here,  by  no  means  a definitive  list,  promote  course  communications  while  encouraging  student  progress 
monitoring,  all  important  for  building  a learning  community  of  self-regulating  students. 


Figure  6.  Assignment  files  named  with  student  last-first  initial  and  assignment  due  date 
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Abstract:  Online  professional  development  offers  faculty  opportunities  to  keep  abreast 
with  evolving  technology  developments  in  times  of  tightening  budgets  and  hectic 
schedules.  The  College  of  Education  at  the  University  of  Hawaii  has  been  offering 
synchronous  online  webinars  utilizing  ElluminateLive!,  a voice -over-IP  (VOIP) 
conferencing  tool,  which  enables  faculty  to  participate  in  workshop  sessions  from  any 
location  with  an  Internet  connection.  Data  collected  from  these  sessions  has  resulted  in  a 
collection  of  lessons  learned  or  a PRIMER  for  designing  effective  synchronous  online 
professional  development  for  faculty. 


Introduction 

As  technology  increasingly  becomes  an  integral  part  of  every  day  life,  universities  and  their  faculty  strive 
to  keep  abreast  with  evolving  technological  developments.  Traditionally,  face-to-face  (F2F)  workshops  are  a 
common  and  familiar  form  of  professional  development  (PD)  to  help  faculty  learn  about  new  technology.  However, 
F2F  workshops  are  inconvenient  and  often  do  not  happen  as  intended.  Most  faculty  already  feel  overwhelmed  with 
the  pressures  of  having  competing  demands  on  their  time  for  teaching,  research  and  service  activities.  Additionally, 
F2F  workshops  take  many  personnel  hours  to  plan  and  staffing  to  deliver.  The  other  problem  is  that  of  locating 
physical  facilities  to  conduct  these  workshops.  In  these  times  of  budgetary  crunch,  both  personnel  and  physical 
facilities  required  to  organize  F2F  workshops  have  become  scarce  resources. 

In  response  to  these  challenges  the  College  of  Education  at  the  University  of  Hawaii  at  Manoa  (COE)  has 
been  offering  online  faculty  PD  sessions  or  “webinars”  utilizing  the  voice -over-IP  (VOIP)  web  conferencing  tool, 
ElluminateLive!.  In  June  2005,  the  COE  purchased  annual  licensing  for  the  use  of  the  ElluminateLive!  web 
conferencing  system  which  essentially  allowed  it  to  extend  its  classrooms  into  a system  of  virtual  classrooms. 
ElluminateLive!  also  allowed  the  COE  to  port  its  PD  training  to  a distributed  audience  via  webinars  (web  seminar). 
These  synchronous  online  webinars  allow  faculty  the  ability  to  participate  in  workshop  sessions  from  any  location 
with  an  Internet  connection.  The  webinars  attracted  a wider  audience  because  faculty  could  participate  from  the 
comfort  of  their  offices  or  homes. 


Background 

Traditionally,  most  online  courses  are  delivered  predominantly  via  asynchronous  technologies  (Waits  & 
Lewis,  2003).  This  could  be  due  to  the  high  cost  of  as  well  as  the  challenges  involved  with  implementing 
synchronous  technologies.  However,  Repman,  Zinskie,  and  Carlson  (2005)  contend  that  the  use  of  synchronous 
technologies  can  increase  meaningful  interactions  in  online  courses. 

In  addition,  Collis  (1996)  presented  four  major  benefits  of  using  synchronous  systems  for  instruction  - 
motivation,  good  feedback,  telepresence  and  pacing.  According  to  Collis,  synchronous  systems  motivate  learners  to 
keep  up  with  their  peers  and  provide  quick  feedback  as  well  as  facilitate  group  decision-making  activities. 
Additionally,  the  real-time  interactions  provided  by  synchronous  systems  encourage  online  community  building  and 
discipline  in  learning  as  well  as  help  students  prioritize  their  learning. 

While  F2F  delivery  has  been  a traditional  method  for  faculty  PD  workshops,  in  times  of  tightening  budgets 
and  over-extended  schedules  new  strategies  are  needed  to  encourage  faculty  to  participate  in  voluntary  PD  sessions. 
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The  compelling  advantages  of  a synchronous  system  make  it  an  ideal  mode  of  delivery  for  faculty  PD  workshops. 
Recognizing  this,  the  COE  has  been  offering  online  synchronous  PD  session  for  its  faculty  utilizing  the  voice-over- 
IP  (VOIP)  web  conferencing  tool,  ElluminateLive ! 


Method 

From  Fall  2005  to  Spring  2007,  the  COE  organized  fifteen  webinars  which  were  attended  by  a total  of  224 
participants.  Example  titles  from  the  webinar  series  include  “Podcasting  for  Educators,  “Open  Source  Software: 
Introducing  OpcnOftice.org'’,  “The  Changing  Landscape  of  Library  Resources,”  and  “Making  the  Shift  to  Online 
Education  - Panel  Discussion”  among  others.  Webinars  typically  lasted  an  hour  and  took  place  on  weekday 
mornings  when  faculty  were  more  often  available.  While  expressly  advertised  to  the  COE  faculty,  COE  students  or 
faculty  expressing  interest  from  other  colleges  were  allowed  to  participate. 

The  delivery  tool  ElluminateLive!  uses  VOIP  technology  to  create  a synchronous  “classroom”  in  which 
participants  hear  and  speak  to  each  other  in  real  time  and  can  view  and  interact  on  a whiteboard.  The  tool  also 
features  a text  chat  window.  When  using  ElluminateLive!,  COE  webinars  typically  feature  a presenter  introducing  a 
new  technology  or  technique  through  a mix  of  web  tours,  PowerPoint  slides  displayed  on  the  whiteboard,  and 
sharing  of  their  own  desktop  for  demonstration.  Each  webinar  is  recorded,  archived  and  made  available  online  for 
access  or  review  at  a later  date  through  an  online  archive. 

Webinars  were  organized  by  the  College’s  instructional  designer  responsible  for  faculty  professional 
development.  In  preparation  for  each  webinar,  the  designer  was  responsible  for  a number  of  organizational  tasks 
including  identifying  and  inviting  guest  presenters,  advertising  the  webinars  through  the  COE  website  and  faculty 
email  list,  handling  email  registrations,  and  doing  a practice  run  with  the  presenters.  During  the  webinar,  the 
designer  acts  as  a host  and  moderator  by  facilitating  introductions,  describing  the  session's  protocol,  relaying 
participants’  questions  from  the  chat  window  to  the  presenter  and  keeping  the  session  on  time.  After  the  webinar  the 
designer  points  participants  to  an  online  post-webinar  survey  evaluation  and  sends  the  link  to  the  session's  archived 
recording  to  the  College  via  email. 


Results 

Webinar  evaluation  survey  data  revealed  that  faculty  highly  rated  items  referring  to  ease  of  use,  interface 
and  interactivity  when  evaluating  ElluminateLive!.  As  shown  in  Table  1,  one  hundred  percent  of  participants 
reported  that  overall  they  like  using  the  Elluminate  Live!  system. 

Table  1.  Percentages  of  webinar  participants  who  agreed  or  strongly  agreed  on  evaluation  items  (n=224) 


Item 

This  webinar  has  motivated  me  to  try  what  I learned.  95% 

The  material  covered  in  this  webinar  is  useful  to  me,  personally.  87% 

The  material  covered  in  this  webinar  is  useful  to  me,  professionally.  94% 

I am  satisfied  with  the  knowledge  and  skills  gained  from  this  workshop.  94% 

I find  the  Elluminate  Live!  system  easy  to  use.  98% 

The  interface  of  this  system  is  pleasant.  97% 

I find  the  system  to  be  very  interactive.  98% 

Overall,  I like  using  the  Elluminate  Live!  system. 100% 


In  addition,  faculty  appreciated  the  physical  convenience  of  the  webinars  as  evidenced  by  the  following 

quotes: 

“This  type  of  session  is  helpful  to  get  a sense  of  what’s  out  there  while  not  venturing  too  far  from  one’s  own 
office.  I appreciate  the  access  and  convenience  of  the  technology.” 

“I  had  a chance  to  participate  in  my  own  office  rather  than  go  to  another  classroom.  That  convenience  was 
great  and  like  students  taking  advantage  of  online  technology,  I could  do  the  same.” 

They  also  positively  commented  on  the  convenience  of  the  webinars  being  recorded  and  archived: 

“I  plan  to  go  through  the  presentation  more  thoroughly,  so  I am  glad  the  webinar  is  archived.” 

I appreciate  being  able  to  enter  half  way  through  the  webinar  and  listen  into  the  discussion  and  then  go 
back  to  replay  the  archived  presentation” 
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By  organizing  these  webinars,  the  COE  leveraged  the  ElluminateLive ! system  to  expand  and  extend  its  PD 
offerings  to  its  constituents,  not  only  in  Hawai‘i  but  throughout  the  Pacific: 

“This  is  such  an  important  connection  for  us  to  be  able  to  connect  via  Elluminate  for  training  sessions 
while  we  are  in  Am  Samoa.  It  means  a great  deal  to  us  to  be  included!  Thanks.” 

When  examining  participants’  reactions  to  the  webinar  sessions  for  specific  indicators  of  success, 
categories  of  best  practices  became  evident.  The  categories  and  examples  of  corresponding  comments  are  listed  in 
Table  2.  Category  development  and  further  consideration  of  evaluation  data  led  to  the  development  of  the  PRIMER 
for  designing  effective  synchronous  online  professional  development  for  faculty. 

Table  2.  Participant  comments  in  categories  of  best  practice 

Planning  and  Organization 

“Great  job  coordinating  this  event” 

“Great  presentation.  Very  well  organized” 

Preparation  and  Rehearsal 

“Chris  [presenter]  was  well-prepared,  and  the  session  was  paced  just  right,  even  for  a beginner  like 

me.” 

Moderation  and  Interaction 

“Thank  you  for  this  informative  and  interactive  presentation.  The  examples  and  demonstrations  on 
the  exemplary  web  course  were  excellent  - it  has  me  wanting  to  re-vamp  my  online  courses.  Great 
ideas” 

“Donna  [presenter]  and  Peter  [moderator]  did  a good  job  of  enabling  the  participants  to  use  it 
[Elluminate  Live!]  actively” 

“Great  job  on  the  podcasting  webinar!  It  was  very  organized  and  the  presentation  went  through 
smoothly.  I’ve  seen  a few  online  workshops  that  experienced  a lot  of  technical  problems  or  there 
was  a long  delay  in  starting  the  workshop.  But  today’s  webinar  was  great!  It  started  right  on  time. 

I also  liked  the  orientation  that  Peter  gave  in  the  beginning  of  the  webinar.  I found  it  very  useful 
and  informative  because  this  was  the  first  time  that  I have  participated  in  a webinar  on  Elluminate” 

“The  introductory  comments/orientation  about  using  Elluminate  and  the  individual  assistance  for 
new  users  was  very  helpful" 


Lessons  Learned/Recommendations 

The  PRIMER  for  designing  effective  synchronous  online  professional  development  for  faculty  has  emerged 
from  the  examination  of  data  collected  from  fifteen,  systematically  designed  COE  webinar  sessions  delivered  over 
two  years. 

PRIMER  is  an  acronym  for  Plan,  Rehearse,  Interact,  Moderate,  Edit  and  Repeat.  Based  on 

• Plan.  There  are  a number  to  planning  techniques  that  can  make  a presentation  go  more  smoothly.  Discuss 
with  presenters  how  synchronous  time  might  be  spent  most  efficiently.  Some  have  found  it  beneficial  to 
give  “homework"  prior  to  a session  to  cut  back  on  time  spent  presenting  content  and  allowing  for  more 
time  for  discussion.  Design  and  distribute  a “Online  Session  in  Progress:  Please  Do  Not  Disturb”  sign  as  a 
pdf  file  that  participants  can  post  on  their  door  when  in  the  session.  Encourage  participants  to  log  in  15 
minutes  early  to  test  their  audio  connections  prior  to  the  session. 

• Rehearse.  Rehearse  all  sessions  regardless  of  how  savvy  the  presenter  may  be  with  the  technology  or 
expert  they  may  be  with  the  content.  Rehearsing  a presentation  with  the  specific  technology  that  will  be 
used,  for  example  the  computer  from  which  the  presenter  will  be  broadcasting,  gives  an  accurate  picture  of 
how  the  tools  will  function  “live”  as  well  as  the  length  of  the  presentation.  Recording  the  presentation  and 
providing  the  presenter  with  access  to  the  recording  allows  them  to  evaluate  aspects  such  as  timing  and 
voice  from  an  objective  standpoint. 

• Interact.  Incorporate  as  much  interactivity  into  presentations  as  possible  to  keep  participants  engaged. 
Recommendations  include  posing  questions  and  using  the  polling  tool  to  display  results,  encouraging  audio 
and  chat  comments  from  the  audience,  and  using  the  whiteboard  text  and  drawing  tools  for  collaboration 
during  a session. 
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• Moderate.  Making  presenters  and  participants  comfortable  in  any  new  tool  is  critical.  As  a session 
organizer  act  as  a “broadcast  host”  and  welcome  participants  as  they  enter  the  session,  ask  each  to  test  their 
audio  and  troubleshoot  with  them  prior  to  the  session’s  commencement,  and  keep  participants  abreast  of 
when  session  will  begin.  Introduce  the  session  and  explain  behavioral  protocol.  For  example,  “Questions 
are  welcomed  throughout  the  presentation.  However  in  order  to  respect  everyone’s  time,  please  type  your 
questions  in  the  chat  window  and  they  will  be  answered  at  the  end  of  the  presentation.”  Track  questions  as 
they  appear  in  the  chat  and  relay  them  to  the  presenter  at  the  end  of  the  session.  Discuss  protocol  with 
presenter  prior  to  session  and  encourage 

• Edit.  Presentations  can  feel  quite  lengthy  for  participants.  Advise  presenters  to  edit  their  presentations  to 
their  most  concise  versions  in  order  to  keep  presentations  short.  One  hour  has  been  a successful  amount  of 
time  for  professional  development.  Start  and  end  sessions  on  time.  Presenters  and  participants  appreciate 
your  respect  for  the  time  of  those  who  have  arrived  on  time. 

• Repeat.  As  with  any  instructional  design  process,  consistency  is  important.  Following  these  steps  for 
every  presentation  will  assist  with  making  it  a success. 


Conclusion 

Online  PD  offers  a number  of  solutions  to  the  challenge  of  declining  budgets  and  time  constraints,  however 
as  a relatively  new  PD  method,  knowledge  of  how  to  effectively  design  for  this  delivery  medium  cannot  be 
assumed.  Practitioners  need  practical,  tested  knowledge  to  complement  their  instructional  design  process  when 
designing  for  online  environments.  It  is  our  sincere  hope  that  the  PRIMER,  a collection  of  lessons  learned  for 
designing  effective  synchronous  online  PD  for  faculty  will  assist  others  in  the  design  and  development  of  successful 
online  faculty  PD  programs. 
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Introduction 

In  2002,  U.S.  Department  of  Education  published  Visions  2020  (U.S.  Department  of  Commerce,  2002),  an 
anthology  of  articles  describing  educational  leaders’  visions  of  how  advanced  technologies  may  tr  ansform 
education.  The  articles  depict  a picture  of  children  engaged  in  problem-  and  project-based  learning  in  authentic, 
technology-enhanced  virtual  environments.  Today,  these  visions  still  seem  far-fetched  science  fiction  to  many 
teachers.  Despite  the  investment  in  education  technology  (U.S.  Department  of  Commerce,  2002),  current  use  of 
technology  in  many  schools  is  still  limited  to  the  application  of  basic  computer  tools  to  support  traditional  approach 
of  teaching  (Bauer  & Kenton,  2005;  Cuban,  Kirkpatrick,  & Peck,  2001;  Ertmer,  2005).  U.S.  Department  of 
Education  is  calling  for  a drastic  change  in  schools  to  fulfill  the  vision.  One  of  the  action  steps  in  the  National 
Education  Technology  Plan  2004  (The  United  States  Department  of  Education,  2005)  is  teacher  training.  As  Ertmer 
(2005)  argues,  many  of  the  conditions  for  technology  integration  already  exist,  the  final  barrier  to  technology 
integration  are  teachers’  beliefs.  Teachers  have  already  developed  a stable  system  of  knowledge  and  beliefs  on 
teaching  and  learning  upon  entering  college  (Pajares,  1992).  They  view  teaching  as  a process  in  which  teachers  pass 
on  knowledge  for  students  to  memorize  (Brookhart  & Freeman,  1992;  Richardson,  1996;  Wideen,  Mayer-Smith,  & 
Moon,  1998).  This  belief  guides  their  future  teaching  practices  (Richardson,  Anders,  Tidwell,  & Lloyd,  1991)  and 
prevents  them  from  adopting  a constructivist  view  of  technology  integration. 

In  this  paper,  we  report  an  analysis  of  teacher  candidates’  knowledge  development  in  a pedagogical 
laboratory.  The  pedagogical  laboratory  is  a setting  where  teacher  candidates  develop  and  try  out  innovative 
technology  integration  curriculum  with  the  help  of  expert  teachers  (Brandsford,  Pellegrino,  & Donovan,  1999). 
Findings  from  this  study  may  inform  the  design  of  similar  programs  and  contribute  to  the  understanding  of  how 
student-centered,  innovative  instructional  activities  may  affect  teacher  candidates’  knowledge  and  beliefs. 

Technological  Pedagogical  Content  Knowledge 

Technological  Pedagogical  Content  Knowledge  (TPACK)  provides  a framework  to  analyze  the  types  of 
knowledge  needed  for  technology  integration  (Mishra  & Koehler,  2006).  TPACK  builds  on  the  work  of  Shulman 
(1987),  who  advanced  the  thinking  of  teacher  knowledge  by  introducing  the  idea  of  pedagogical  content  knowledge 
as  a key  type  of  teacher  knowledge.  Traditional  notion  of  teacher  knowledge  focuses  on  content,  pedagogy,  and  the 
intersection  of  the  two.  TPACK  introduces  another  element,  technology,  into  the  picture.  This  framework 
categorizes  teacher  knowledge  into  seven  types:  Content  Knowledge  (CK),  Pedagogical  Knowledge  (PK), 
Technology  Knowledge  (TK),  Pedagogical  Content  Knowledge  (PCK),  Technological  Content  Knowledge  (TCK), 
and  Technological  Pedagogical  Content  Knowledge  (TPACK).  CK  is  the  knowledge  of  the  subject  matter  per  se.  PK 
includes  general  principles  and  strategies  related  to  classroom  management  and  organization  applicable  to  all  subject 
areas.  TK  is  typically  the  type  of  knowledge  emphasized  in  training  related  to  technology  integration,  which  focuses 
on  teaching  teachers  to  use  various  computer  hardware  and  software.  PCK  refers  to  the  specific  strategies  that  can  be 
employed  to  make  certain  content  comprehensible  to  learners.  TCK  is  at  the  intersection  of  technology  and  content. 
It  describes  how  specific  subject  matter  is  changed  by  technology.  TPK  refers  to  the  knowledge  of  the  affordances 
of  different  technological  tools  and  generally  how  they  should  be  used  in  teaching.  TPACK  is  pedagogical 
knowledge  in  the  context  of  specific  content  and  technologies.  A teacher  who  is  good  at  technology  integration 
knows  how  to  leverage  the  affordances  of  particular  technologies  to  teach  certain  difficult  concepts. 
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TPACK  has  been  used  as  a framework  to  understand  teacher  knowledge  development  in  the  context  of 
designing  technology-based  lessons.  It  served  as  the  codes  for  analyzing  observation  notes  of  activities  and 
interactions  of  design  teams  (Koehler,  Mishra,  & Yahya,  2007)  and  provided  the  structure  for  a survey  tool  created 
to  quantitatively  assess  teachers’  knowledge  development  (Koehler  & Mishra,  2005). 

Teacher  Learning  in  a Technology-Enhanced  Pedagogical  Laboratory 

A technology-enhanced  pedagogical  laboratory  was  set  up  in  a teacher  education  program  in  a southern 
research/teaching  university  to  impact  teacher  candidates'  knowledge  and  beliefs  related  to  technology  integration 
(Ma,  Williams,  Prejean,  Lai,  & Ford,  2008).  The  pedagogical  laboratory  emphasizes  three  key  components  found 
important  in  the  teacher  learning  literature,  including  experience,  reflection  and  support.  Teacher  candidates  gain 
personal  experiences  in  practicing  research-based  instructional  strategies  and  acquire  vicarious  experiences  by 
observing  the  teaching  practice  of  their  peers  and  expert  teachers.  Reflection  is  another  key  element.  Teacher 
candidates  are  encouraged  to  reflect  on  their  experiences.  They  are  guided  to  think  of  themselves  as  scientists,  who 
conduct  experiments  and  reflect  on  the  results  and  changes  needed.  This  mentality  may  help  them  become  reflective 
practitioners.  In  addition,  instructional  materials  and  expert  teachers  are  available  in  the  pedagogical  laboratory  to 
provide  support  to  teacher  candidates. 

A series  of  studies  have  been  conducted  to  evaluate  the  impact  of  the  pedagogical  laboratory  experience  on 
teacher  candidates’  knowledge  and  beliefs.  In  a qualitative  study  of  teacher  candidates’  reflective  journals  and 
interviews  (Ma,  Lai,  Williams,  Prejean,  & Ford,  2008),  the  pedagogical  laboratory  experience  was  deemed 
extremely  valuable  by  teacher  candidates.  It  gave  them  a new  understanding  of  and  inspiration  for  teaching.  Teacher 
candidates  realized  how  difficult  but  also  exciting  and  rewarding  teaching  can  be.  The  field  experience  also 
challenged  some  of  their  beliefs  of  teaching  and  technology  integration. 

In  a quantitative  study  (Ma,  Lai,  Williams,  & Prejean,  2008),  an  instrument.  Teachers’  Beliefs  Regarding 
Technology  Use  Survey  (TBTUS),  was  employed  to  measure  the  impact  of  the  pedagogical  laboratory  experience 
on  teacher  candidates’  beliefs  regarding  technology  integration.  Analysis  of  TBTUS  indicates  that  the  impact  of  the 
pedagogical  laboratory  experience  was  largely  insignificant.  The  qualitative  data  suggest  that  changes  might  be 
incremental  and  TBTUS  might  not  be  sensitive  to  the  changes  that  occurred  after  22-hour  of  treatment,  with  only  six 
hours  of  real  teaching  experience. 

In  a more  recent  study,  researchers  expanded  the  treatment  time  from  four  weeks  to  a 15-week  semester 
(Ma,  Williams,  Ford,  & Prejean,  2009).  The  treatment  includes  not  only  the  pedagogical  laboratory  experience,  but 
also  skill  development  and  lesson  idea  development  for  various  software  applications,  presentation  of  theories 
related  to  student-centered  learning,  and  case  study  analysis  of  how  teachers  use  specific  software  applications  in  the 
classroom.  Statistical  analysis  indicates  that  the  pedagogical  laboratory  had  statistically  significant  impact  on  the 
following  beliefs:  learner-centered  beliefs  about  learners,  learning,  and  teaching  (LB-LLT),  self-efficacy  beliefs 
about  technology  integration  (SEB),  beliefs  about  perceived  value  (PV)  of  computers  for  instructional  purposes. 

Methods 


Research  Purpose  and  Questions 

Previous  research  shows  that  a technology  integration  course  that  includes  a 4-week  pedagogical  laboratory 
experience  was  effective  in  affecting  changes  in  teacher  candidates’  beliefs.  Yet,  the  impact  of  the  pedagogical 
laboratory  experience  itself  was  insignificant  on  teacher  beliefs  as  measured  by  TBTUS.  However,  qualitative  data 
indicates  that  teacher  candidates  did  benefit  from  the  pedagogical  laboratory  experience.  To  gain  a better 
understanding  of  what  teacher  candidates  learned  from  pedagogical  laboratory  experience,  we  use  TPACK  as  a 
framework  to  analyze  teacher  candidates’  reflective  writing  after  the  field  experience.  The  following  two  questions 
are  addressed  in  the  analysis: 

1.  How  does  the  pedagogical  laboratory  experience  contribute  to  teacher  candidates’  knowledge  regarding 
technology  integration? 

2.  Does  teaching  different  lessons  (digital  storytelling  vs.  robotics)  in  the  pedagogical  laboratory  contribute 
differently  to  teacher  candidates'  knowledge  regarding  technology  integration? 
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Subjects 


Forty-eight  teacher  candidates  from  four  sessions  of  a technology  integration  course  participated  in  the 
study.  Twenty-eight  of  the  pre-service  teachers  majored  in  early  childhood  education  and  20  of  the  teacher 
candidates  majored  in  elementary  education.  Each  participating  teacher  candidates  wrote  two  reflective  journals. 

Procedures 

The  pedagogical  laboratory  experience  took  place  in  the  middle  of  the  semester.  It  included  three  phases: 
teacher  candidate  preparation,  laboratory  experience,  and  reflective  journal  writing.  It  took  teacher  candidates  a total 
of  22  hours  over  four  weeks  to  complete.  Teacher  candidate  preparation  lasted  three  and  a half  weeks,  with  a total  of 
approximately  14  hours.  Laboratory  experience  took  place  on  two  consecutive  Saturdays  and  lasted  a total  of  six 
hours.  Reflective  journal  writing  took  about  two  hours. 

Teacher  candidate  preparation  consisted  of  three  main  components.  First,  the  university  instructor  modeled 
the  teaching  of  a robotics  lesson  to  elementary  education  majors  and  a digital  storytelling  lesson  to  early  childhood 
education  majors.  The  second  component  of  teacher  candidate  preparation  focused  on  providing  vicarious 
experiences  and  pedagogical  knowledge  needed  to  affect  teacher  candidates’  beliefs.  Video  case  studies  of  project- 
based  learning  and  cognitive  apprenticeship  were  presented,  and  teacher  candidates  were  encouraged  to  compare 
these  student-centered  learning  environments  with  traditional  classrooms  to  identify  the  rationale  for  and 
characteristics  of  student-centered  learning.  A list  of  facilitation  strategies  were  presented  to  teacher  candidates.  The 
list  includes  not  only  general  strategies  such  as  questioning,  modeling  and  providing  motivational  prompts,  but  also 
specific  strategies  to  encourage  reflection  and  guide  group  collaboration.  Teacher  candidates  were  required  to  watch 
video  case  studies  from  INTIME  (2001)  website  and  identify  what  strategies  teachers  used  to  facilitate  the 
technology-enhanced  lessons.  The  third  component  involved  collaborative  lesson  planning.  As  a team,  teacher 
candidates  planned  a series  of  robotics  or  digital  storytelling  activities  for  the  children.  Sample  lessons  were 
provided  to  guide  lesson  planning.  Teacher  candidates  were  required  to  consider  the  following  in  the  planning 
phase:  state  and  national  standards,  the  children’s’  grade  levels,  lesson  procedures,  and  possible  facilitation 
strategies. 

During  the  laboratory  experience,  every  two  teacher  candidates  were  paired  to  facilitate  a student-centered 
lesson  to  a group  of  one  or  two  children  for  two  three-hour  field  experience  sessions. 

During  the  break  of  the  robotics  program,  teacher  candidates  took  students  outside  to  complete  an 
educaching  activity.  Educaching  is  the  use  of  goecaching  to  achieve  educational  goals.  Geocaching  is  an  outdoor 
adventure  game  where  individuals  use  Global  Position  System  (GPS)  devices  to  find  hidden  caches  (Groundspeak 
Inc.,  2009).  An  educaching  activity  was  added  to  the  robotics  camp,  because  the  robotics  challenges  were  intense  for 
both  the  students  and  candidates;  an  educaching  activity  provided  a change  of  pace  and  gave  students  an  opportunity 
to  take  a break  outside. 

Teacher  candidates  wrote  a reflective  journal  after  each  three -hour  field  experience  session.  They  were 
required  to  think  of  a critical  incident  that  happened  during  their  field  experience  to  anchor  their  reflection.  A critical 
incident  can  be  identified  by  thinking  of  an  “aha”  or  “oops”  moment  that  teacher  candidates  experienced  during  the 
field  experience.  It  is  typically  a significant  moment  that  may  raise  some  questions  or  challenge  one’s  beliefs.  A 
three-step  process  display  scaffold  was  used  to  guide  the  writing  of  reflective  journals.  In  previous  research,  this 
scaffold  significantly  enhanced  the  teacher  candidates’  reflective  journal  writing  (Lai  & Calandra,  in  press). 

Data  Analysis 

A total  of  94  reflective  journals  were  analyzed  by  using  TPACK  as  a framework  for  coding.  Each  journal 
was  assigned  a descriptive  code,  summarizing  the  key  idea  that  teachers  learned  from  the  critical  incident  occurred 
during  the  field  experience.  The  codes  were  grouped  by  themes  and  then  into  categorical  codes  representing  the 
different  knowledge  types  including  CK,  TCK,  TK,  PCK,  PK,  TPK,  and  TPCK.  The  frequencies  of  the  categorical 
knowledge  type  codes  were  entered  into  Excel  to  create  a graphical  representation  so  as  to  identify  the  patterns  of 
teacher  candidates’  knowledge  gains. 


Findings 

Question  1:  How  does  the  pedagogical  laboratory  experience  contribute  to  teacher  candidates’  knowledge  regarding 
technology  integration? 

Tables  1 shows  that  the  pedagogical  laboratory  contributed  to  the  following  types  of  knowledge 
for  teacher  candidates  who  facilitated  the  digital  storytelling  lesson:  PK,  PCK,  TPK,  and  TPCK.  Close  to 
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half  (47%)  of  the  reflections  focused  on  PK.  Among  these  reflections,  about  half  of  them  were  concerned 
with  ensuring  equal  participation  from  the  children.  In  these  reflections,  one  child  in  the  group  played  a 
dominant  role,  and  the  other  child  tended  to  be  shy.  Student  teachers  tried  to  give  both  children  in  the  group 
an  equal  opportunity  to  participate  in  the  storytelling  activity.  They  reflected  on  the  importance  of  this 
strategy.  Some  of  them  discussed  the  scenario  at  the  personal  level.  A couple  of  student  teachers 
sympathized  with  the  children  who  were  shy.  One  of  them  also  tried  to  control  her  feelings.  She  wrote, 

I sympathized  maybe  more  than  I should  have  with  her  and  I found  myself  trying  to  connect  with 
her  more  than  the  other  child,  which  as  a teacher  I know  I can’t  do,  I can't  wear  my  feelings  on  my 
sleeve  and  I have  to  be  able  to  treat  my  children  equally.” 

Five  of  the  reflections  focused  on  strategies  for  managing  inappropriate  behaviors  such  as  conflict  between  children, 
lack  of  respect  to  each  other,  and  too  much  joking  and  chatting  between  the  children.  Three  reflections  discussed 
how  a lesson  may  not  run  as  planned  and  how  teachers  need  to  be  flexible  in  changing  the  plan.  Other  reflections 
were  concerned  with  giving  children  control  and  allow  them  to  work  at  their  own  pace,  working  with  other  teachers 
who  did  not  want  children  to  handle  equipment,  working  with  children  whose  English  is  not  the  first  language,  and 
managing  time  to  complete  work  before  the  lesson  ended. 

Less  than  a quarter  (22%)  of  the  journals  reflected  on  TPK.  Seven  reflections  discussed  the  affordances  of 
voice  recording  and  movie  making  in  engaging  children.  A candidate  wrote  about  her  experience  working  with  two 
kindergarten  children, 

I found  that  the  kids  participated  most  when  they  could  actually  see  something  or  hear  something 
as  a result  of  their  input.  So  I believe  that  when  using  technology,  it  works  best  if  you  can 
physically  see  or  hear  results.  Because  when  they  came  to  writing  a script  or  coming  up  with  a 
storyline,  the  children  weren't  as  involved.  Once  it  came  to  taking  pictures  and  recording  their 
voices,  they  were  able  to  see  how  it  all  came  together. 

Candidates  also  reflected  on  the  importance  of  giving  children  control  when  using  technology. 

About  one  fifth  (18%)  of  the  reflections  discussed  PCK  related  to  story  development.  Seven  reflections 
discussed  the  strategies  for  guiding  story  development.  Generating  an  idea  for  the  story  was  a challenging  task  for 
many  groups.  Student  candidates  helped  children  develop  story  ideas  by  asking  open-ended  questions  and  analyzing 
sample  stories.  They  learned  that  to  help  children  generate  an  idea,  it  is  important  to  “constantly  interact  with  the 
children  and  to  get  down  to  their  level.”  They  found  that  “children  learn  a lot  from  modeling”  and  although  story 
development  was  challenging,  “with  patience  and  gentle  prompting,  kids  can  collect  their  thoughts  and  create 
something  special.”  Five  reflections  discussed  strategies  for  helping  children  to  compromise  during  story 
development.  For  groups  consisting  of  a boy  and  a girl,  it  was  especially  challenging  when  the  two  children  had 
totally  different  ideas  for  stories  and  would  not  compromise.  They  found  that  getting  children  to  play  with  each 
other  helped  them  get  to  know  one  another  and  made  it  easier  for  them  to  compromise. 

Thirteen  percent  of  the  reflections  dealt  with  TPCK.  They  focused  on  strategies  for  teaching  digital 
storytelling.  To  work  with  young  children  who  cannot  read  or  write,  they  discussed  strategies  such  as  having 
children  draw  pictures  for  each  scene  of  the  storyboard.  They  realized  that  the  standard  process  of  writing  the  script 
first  and  taking  pictures  second  might  not  always  work  for  digital  storytelling.  For  children  who  already  had  a story 
idea,  allowing  them  to  take  and  view  some  pictures  first  actually  facilitated  story  development.  A candidate  wrote, 
“My  action  in  a similar  situation  would  be  to  evaluate  the  students.  If  they  seem  to  be  able  to  handle  doing  things 
differently,  and  can  still  stay  on  task,  I would  handle  the  situation  the  same  way.”  She  reflected  further,”  it  is 
sometimes  better  to  allow  the  student  to  lead  you.  By  allowing  them  to  provide  input,  and  responding  to  that  input 
positively,  they  felt  as  though  we  considered  their  ideas  important.” 
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Knowledge  Types 

Themes 

Number  of 
reflections 

CK:  0 

0 

TCK:  0 

0 

TK:  0 

0 

Dealing  with  change  of  plan 

3 

Ensuring  equal  participation 

11 

PK:  26,  47% 

Giving  children  control 

1 

Managing  inappropriate  behaviors 

5 

Engaging  children 

3 

Working  with  other  teachers 

1 

Managing  time 

1 

Working  with  ESL  children 

1 

PCK:  10,  18% 

Guiding  story  development 

7 

Guiding  students  to  compromise  on  story  development 

3 

TPK:  12,  22% 

Affordances  of  voice  recording  in  engaging  children  and 
providing  ownership 

7 

Giving  children  control  in  using  technologies 

5 

TPCK:  7,  13% 

Strategies  in  teaching  digital  storytelling 

7 

Table  1:  Coding  summary  for  teacher  candidates  who  facilitated  the  digital  storytelling  lesson 

Table  2 shows  that  teacher  candidates  who  facilitated  the  robotics  lesson  also  gained  knowledge  in  the 
following  categories:  PK,  PCK,  TPK,  and  TPCK.  About  38%  of  the  reflections  were  related  to  PK.  Five  of  the 
reflections  focused  on  ensuring  equal  participation  from  both  children  in  the  group.  They  cautioned  themselves  that 
they  should  not  judge  students  based  on  whether  they  wanted  to  take  over  or  not. 

One  thing  I learned  was  that  students  often  take  over  and  don't  even  realize  it.  Student  one  wasn't 
doing  all  the  work  on  purpose  and  student  two  wasn't  being  lazy.  It  was  the  fact  that  both  of  these 
students  were  fully  capable  and  one  was  simply  more  interested  in  the  other.  With  this  reason  in 
mind.  I had  to  realize  that  sometimes  you  need  to  nicely  let  students  know  to  take  turns  and 
encourage  them  to  work  together. 

Three  reflections  focused  on  addressing  inappropriate  behaviors  such  as  getting  off  track,  having  conflict  with  each 
other,  and  failing  to  follow  safety  instructions.  Two  reflections  were  concerned  with  the  strategies  for  dealing  with 
interruptions  from  observers.  Another  two  reflections  discussed  how  teaching  the  lesson  improved  candidates’ 
knowledge  about  the  learners.  They  were  amazed  how  students  developed  a scavenger  hunt  game  on  their  own  after 
completing  educaching.  Writing  the  reflections  provided  candidates  with  an  opportunity  to  assess  their  own 
knowledge  of  robotics.  Two  reflections  described  how  they  were  intimidated  by  children's  ability  and  felt 
unprepared.  They  discussed  the  need  to  better  prepare  themselves,  but  also  realized  that  sometimes  children  know 
more  than  teachers  and  teachers  can  still  ask  probing  questions  to  lead  students  into  deeper  discussion.  One  journal 
reflected  on  the  strategy  of  constantly  monitoring  time  spent  for  the  project  and  adjusting  activities  accordingly. 
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Knowledge  Types 

Themes 

Number  of 
reflections 

CK:  0 

0 

TCK:  0 

0 

TK:  0 

0 

PK:  15,  38% 

Dealing  with  interruption  from  observers 

2 

Ensuring  equal  participation 

5 

Knowledge  of  children's  ability 

2 

Managing  inappropriate  behaviors 

3 

Managing  teachers  feeling  of  inadequacy 

2 

Managing  time 

1 

PCK:  2,  5% 

Teaching  mathematics 

1 

Working  with  a child  who  was  engaged  in  the  storyline 

1 

TPK:  5,  13% 

Affordances  of  microscope  and  geocaching  in  engaging 
children 

5 

TPCK:  17,  44% 

Strategies  to  guide  students  to  program  the  robot 

14 

Engaging  children  in  the  robotics  activity 

3 

Table  2:  Coding  summary  for  teacher  candidates  who  facilitated  the  robotics  lesson 

About  44%  of  the  reflections  focused  on  TPCK.  Fourteen  of  the  reflections  dealt  with  strategies  for  guiding 
students  to  program  the  robot.  Teacher  candidates  learned  that  when  students  encounter  a programming  problem,  it 
is  important  to  allow  student  control  and  let  them  try  out  their  solutions.  Teachers  should  provide  guidance  for  their 
experiments  and  encourage  students  to  learn  from  failures.  One  candidate  reflected, 

I think  this  also  goes  back  to  science  and  the  scientific  method.  The  students  must  form  a hypothesis  and 
test  it.  They  are  then  taught  that  testing  and  retesting  is  important  for  learning  new  concepts.  I learned  that 
students  learn  best  from  performing  tasks  themselves. 

They  also  practiced  unique  strategies  such  as  designing  a dance  for  the  robot  by  actually  having  the  children  in  the 
group  dance  themselves.  This  strategy  was  motivating  and  fun  to  the  children.  Student  teachers  reflected  that  it  is 
important  to  be  creative  and  to  try  “a  variety  of  teaching  strategies”  in  the  lesson. 

Thirteen  percent  of  the  reflections  were  concerned  with  TPK.  They  were  accounts  of  affordances  of 
microscope  and  geocaching.  The  robotics  activity  involves  the  use  of  microscope.  Teacher  candidates  found  that 
having  children  use  the  microscope  in  the  middle  of  the  robotics  activity  re-engaged  them  when  they  were  tired. 

Five  percent  of  the  reflections  were  related  to  PCK.  One  reflection  discovered  that  there  were  different 
ways  of  figuring  out  a mathematics  problem;  teachers  should  allow  children  time  to  explore  the  problem  themselves 
before  giving  out  the  answer.  Another  reflection  discussed  the  importance  of  embedding  a story  in  the  robotics 
activity.  A child  in  the  group  was  pretending  to  be  in  the  situation  described  in  the  story  and  was  motivated  to 
complete  the  tasks. 

Question  2:  Does  teaching  different  lessons  (digital  storytelling  vs.  robotics)  in  the  pedagogical  laboratory  contribute 
differently  to  teacher  candidates'  knowledge  regarding  technology  integration? 

Figure  1 shows  that  both  lessons  contributed  to  teacher  candidates’  knowledge  related  to  PK,  PCK,  TPK, 
and  TPCK.  In  both  lessons,  the  reflections  focused  heavily  on  PK.  In  the  digital  storytelling  lesson,  candidates 
seemed  to  gain  more  knowledge  related  to  PCK  and  TPK;  whereas  in  robotics,  candidates  reflected  more  on  TPCK. 
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Knowledge  Types 
(Digital  Storytelling  and  Robotics) 


PCK  (pedagogical  PK  (pedagogical  TPCK  TPK  (technological 

content  knowledge)  knowledge)  (Technological  pedagogical 

pedagogical  knowledge) 

content  knowledge) 


Figure  1.  Contribution  of  the  digital  storytelling  and  robotics  lessons  to  candidates’  knowledge 


Discussions 

This  study  found  that  the  pedagogical  laboratory  experience  contributed  to  teacher  candidates’  knowledge 
regarding  pedagogy  and  the  intersection  of  pedagogy  with  content  and  technology.  This  is  not  surprising,  because 
the  focus  of  the  pedagogical  laboratory  was  teaching;  candidates  had  to  use  various  pedagogical  strategies  related  to 
teach  specific  content  using  technology.  The  pedagogical  laboratory  might  have  also  enhanced  candidates’ 
knowledge  in  the  area  of  TK,  because  teachers’  self-efficacy  beliefs  related  to  technology  use  improved  significantly 
after  the  semester  in  which  the  pedagogical  laboratory  experience  took  place.  However,  the  reflections  were  not  a 
good  assessment  of  this  type  of  knowledge,  because  the  reflection  writing  scaffold  used  in  this  study  asked 
candidates  to  focus  on  the  critical  incidents  during  the  field  experience  and  these  incidents  tended  to  illicit  reflection 
of  pedagogical  strategies  in  the  context  of  a certain  subject  matter  and  technology  rather  than  content  and  technology 
themselves.  This  study  shows  to  researchers  that  the  reflection  of  critical  incidents  in  real  world  teaching  experience 
may  serve  as  an  effective  assessment  tool  of  teachers’  acquisition  of  PK,  PCK,  TPK,  and  TPCK. 

This  study  confirms  the  value  of  the  pedagogical  laboratory  experience.  The  knowledge  that  teacher 
candidates  gained  is  subtle  and  situational,  especially  those  at  the  intersection  of  pedagogy,  content,  and  technology. 
These  types  of  knowledge  are  difficult  to  teach  without  real  world  teaching  experience.  This  finding  adds  to  the 
argument  in  the  existing  literature  why  personal  experience  in  teaching  with  technology  should  be  an  essential 
element  in  training  teachers  for  technology  integration. 

Interestingly,  although  the  pedagogical  laboratory  experience  was  designed  to  enhance  teachers’ 
knowledge  related  to  technology  integration,  many  of  the  reflections  focused  on  teachers’  knowledge  gains  in 
general  pedagogical  knowledge.  This  is  consistent  with  our  findings  in  previous  semesters.  As  discussed  in  our 
previous  research  (Ma,  Lai,  Williams,  & Prejean,  2008),  candidates  had  little  personal  teaching  experience  in 
practicing  pedagogical  strategies.  Therefore,  common  classroom  management  issues  became  salient  for  them  in  this 
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teaching  experience.  If  teacher  candidates  had  opportunities  to  practice  general  pedagogical  knowledge  such  as 
classroom  management  strategies  prior  to  taking  the  technology  integration  course,  they  might  focus  their  reflection 
more  on  strategies  related  to  technology  and  content. 

This  study  found  that  although  both  robotics  and  digital  storytelling  lessons  contributed  to  PK,  PCK,  TPK, 
and  TPCK,  there  seemed  to  be  some  differences  in  teacher  candidates’  learning  in  the  two  lessons.  Robotics  was 
more  challenging  in  terms  of  technology  and  content.  Teacher  candidates’  reflections  focused  more  on  how  to  guide 
students  to  program  the  robots.  Therefore,  candidates  reflected  more  on  TPCK  after  the  robotics  lesson.  As  for 
digital  storytelling,  developing  a story  idea  was  the  most  challenging  aspect  of  the  project,  so  many  reflections  were 
related  to  PCK.  After  completing  the  difficult  task  of  story  development,  voice  recording  provided  a relief  to 
candidates  because  it  was  more  hands-on  and  engaging.  Therefore,  quite  a few  candidates  reflected  on  the 
affordances  of  this  technology,  which  contributed  to  their  development  of  TPK.  Requiring  candidates  to  teach  a 
variety  of  lessons  that  are  challenging  in  different  aspects  of  pedagogy,  content,  and  knowledge  may  better  prepare 
them  for  technology  integration  in  the  classroom  setting. 

Mapping  candidates’  knowledge  development  in  the  pedagogical  laboratory  not  only  confirmed  the  value 
of  the  experience,  it  may  also  help  us  identify  the  support  that  should  be  provided  to  teacher  candidates.  Offering 
various  types  of  knowledge  of  the  potential  issues  that  previous  teacher  candidates  may  encounter  prior  to  the  field 
experience  may  better  prepare  candidates  to  deal  with  the  issues  and  learn  from  the  experience. 

This  study  indicated  that  there  is  a gap  in  the  types  of  knowledge  that  the  pedagogical  laboratory 
experience  impacted.  In  either  the  robotics  or  the  digital  storytelling  program,  an  important  type  of  knowledge,  TCK 
was  not  addressed  in  candidates'  reflections.  This  is  not  surprising,  because  the  technology  integration  course 
focused  more  on  technology  and  related  pedagogy  rather  than  content.  Faculty  teaching  technology  integration 
courses  may  work  with  faculty  who  teach  content  and  methods  courses  to  help  candidates  develop  TCK  and  TPCK. 
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Abstract 

PictoPal  is  the  name  of  a technology-supported  intervention  designed  to  foster  the  development  of  emergent 
literacy  in  four  and  five  year  old  children.  During  the  iterative  process  of  context  analysis,  prototype  design, 
evaluation,  and  revision,  design  research  has  been  conducted  to  gain  insight  into  desirable  program  characteristics, 
implementation  strategies  and  the  teachers  as  designers  of  curriculum  materials.  To  date,  five  prototypes  have  been 
designed  and  evaluated,  four  of  which  contributed  to  pupil  learning  gains  after  eight  weeks  of  classroom 
intervention.  In  addition  to  the  PictoPal  intervention  itself,  this  study  has  yielded  design  guidelines  for  developing 
and  implementing  software  for  early  literacy.  This  paper  examines  the  research  and  development  processes,  as  well 
as  the  results,  in  light  of  an  existing  model  for  design  research. 

Introduction 

To  explore  how  to  use  and  apply  emerging  theories  on  learning  and  instruction,  design  studies  are 
increasingly  being  used;  these  tend  to  emphasize  the  whole  cycle  of  scientific  inquiry,  often  involving  sub-studies 
and  analysis  cycles  throughout  the  stages  of  problem  identification,  hypothesis  (re)forming,  treatment  development 
and  testing.  Design  studies  require  interaction  and  collaboration  between  researchers,  teachers,  and  other 
stakeholders.  This  paper  speaks  to  a fervent  call  for  design  researchers  to  share  their  emerging  insights  on  how  to 
conduct  design  studies  and  communicate  the  results.  Following  a brief  description  of  design  research,  several 
existing  models  for  design  research  are  discussed;  and  the  model  used  to  describe  this  study  is  presented. 

Thereafter,  each  element  in  the  model  is  illustrated  through  the  PictoPal  case.  The  paper  concludes  with  reflections 
on  the  PictoPal  research  in  particular,  and  design  research  in  general. 

Design  research 


Momentum 

According  to  Barab  and  Squire  (2004),  design  research  is  “a  series  of  approaches,  with  the  intent  of 
producing  new  theories,  artifacts,  and  practices  that  account  for  and  potentially  impact  learning  and  teaching  in 
naturalistic  settings.’'  The  field  of  design  research  has  been  gaining  momentum,  particularly  in  educational  studies, 
over  the  last  decade.  In  special  issues  of  highly  respected  journals,  the  need  for  attention  to  be  given  to  design 
research  was  demonstrated:  Educational  Researcher  (2003,  31(1)),  Journal  of  the  Learning  Sciences  (2004,  13(1)); 
Educational  Psychologist  (2004,  39(4)).  Books  devoted  to  the  topic  examine  design  research  conceptualization  (van 
den  Akker,  Gravemeijer,  McKenney,  & Nieveen,  2006b),  methodological  considerations  (Kelly,  Lesh,  & Baek, 
2008),  and  even  guides  and  tools  for  conducting  design  studies  (Richey  & Klein,  2007).  While  many  of  the 
publications  are  associated  with  the  Science,  Technology,  Engineering  and  Mathematics  (STEM)  disciplines,  one 
volume  looks  specifically  at  applications  in  language  and  literacy  research  (Reinking  & Bradley,  2008).  Across  the 
different  publications  on  design  research,  three  main  motives  for  using  this  approach  are  evident:  Increasing  the 
relevance  of  research;  developing  empirically-grounded  theories;  and  increasing  the  robustness  of  design  practice. 
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Characteristics 

Design  studies  vary  along  with  the  research  and  design  aims.  Some  studies  set  out  to  validate  and  develop 
learning  theories;  others  focus  more  on  developing  practical  solutions  to  educational  problems;  and  yet  others  are 
geared  more  toward  widespread  adoption  of  new  curricula  (Nieveen,  McKenney,  & van  den  Akker,  2006).  Despite 
these  differences,  there  seems  to  be  some  degree  of  consensus  on  characteristics  of  design  research.  Wang  and 
Hannafin  (2005)  describe  design  research  as  pragmatic;  grounded;  interactive,  iterative,  flexible;  integrative;  and 
contextual.  According  to  van  den  Akker,  McKenney  & Nieveen  (2006a),  design  research  may  be  characterized  as 
interventionist;  iterative;  process-oriented;  utility-oriented;  and  theory-oriented.  In  their  book  about  design  research 
in  the  domain  of  literacy.  Reinking  and  Bradley  (Reinking  & Bradley,  2008)  delineate  seven  characteristics  of 
design  research:  intervention  centered;  theoretical;  goal-oriented;  adaptive  and  iterative;  transformative; 
methodologically  inclusive  and  flexible;  and  pragmatic. 

Models 

Several  different  models  for  design  research  have  cropped  up  in  literature.  The  Osmotic  Model , offered  by 
Eljersbo  et  al  (2008),  depicts  the  design  cycle  parallel  to  elements  in  the  research  cycle.  The  authors  point  out  that 
both  cycles  depart  from  the  problem  and  would  ideally  run  simultaneously,  but  state  that  this  ideal  is  often  not  the 
case.  Bannan-Ritland  and  Baek  (2008)  have  developed  the  Integrated  Learning  Design  Framework , which  depicts 
four  main  stages  and  across  those,  14  steps,  in  a combined  approach  to  research  and  development.  Along  with  the 
process  model,  guiding  questions  for  research  and  examples  of  applicable  methods  for  each  main  phase  are  given. 
Reeves  (2000)  offers  a minimalist  model  that  highlights  four  main  phases  of  design  research:  problem  analysis; 
solution  development;  iterative  refinement;  and  reflection  to  produce  design  principles.  He  compares  these  phases 
to  the  four  phases  of  predictive  research.  In  contrast  to  the  aforementioned  three  models,  McKenney,  van  den  Akker 
and  Nieveen  (2006)  offer  model  which  is  more  conceptually-oriented  than  process -oriented.  This  model  depicts 
tenets  guiding  a research  and  development  cycle,  situated  in  a particular  context,  yielding  three  main  outcomes: 
professional  development  of  the  participants;  the  designed  intervention;  and  design  principles.  In  addition  to  these 
visual  models,  Gravemeijer  and  Cobb  (2006)  describe  important  steps  in  the  three  main  phases  of  their  work: 
preparing  for  a design  experiment;  conducting  a design  experiment;  and  retrospective  analysis.  Finally,  Reinking 
and  Bradley  (2004)  pose  six  questions  as  a guide  for  conducting  formative  experiments,  relating  to:  pedagogical 
goals;  classroom  intervention;  factors  affecting  the  intervention;  modifications  to  the  intervention;  unpredicted 
effects  of  the  intervention;  and  changes  in  the  instructional  environment  due  to  the  intervention.  Each  of  these 
questions  is  tied  to  a characteristic  of  design  research,  as  cited  in  the  previous  paragraph. 

Building  on  the  previous  models,  McKenney,  Reeves  and  Herrington  (in  press)  offer  a model  which  depicts 
the  research  and  development  activities  and  outputs,  along  with  how  those  interact  with  practice.  In  this  model  (see 
Figure  1),  the  research  and  development  activities  are  shown  in  the  three  squares;  the  two  outputs  are  the  final 
intervention  and  design  principles;  and  the  interaction  with  practice  is  shown  to  increase  over  time,  as  mature 
designs  warrant  larger  scales  of  implementation  and  diffusion.  We  use  this  model  to  describe  the  design  research 
approach  of  PictoPal.  The  following  section  describes  how  each  element  in  the  model  was  undertaken  in  the 
PictoPal  case. 
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Figure  1 : Design  research  model  (McKenney,  Reeves  & Herrington,  in  press)  used  to  describe  PictoPal 


PictoPal  as  design  research 


Research  and  development 

Phase  1 : Analysis  and  exploration.  Recent  increased  interest  in  early  literacy  has  been  evidenced  by  the  growing 
trend  to  set  state  standards  for  the  literacy  development  of  young  children  (Neuman  & Roskos,  2005)  and  the  call  for 
teacher  professional  development  to  meet  the  corresponding  curricular  demands  (Strickland,  2003).  In  the 
Netherlands  and  abroad,  the  implementation  of  early  literacy  standards  in  educational  practice  proves  demanding  for 
teachers.  At  the  same  time,  teachers  in  general  and  of  this  age  range  in  particular  struggle  to  introduce 
pedagogically  appropriate  technology  applications.  The  PictoPal  study  set  out  to  address  these  problems  through  the 
design  and  development  of  a technology-rich  learning  environment  for  the  development  of  early  literacy  skills  in 
kindergarten.  To  understand  the  problem  better,  a small-scale  needs  and  context  analysis  was  carried  out.  Toward 
generating  ideas  for  solutions,  exploration  of  existing  knowledge,  products  and  interested  parties  was  conducted. 

The  needs  and  context  analysis  was  conducted  to  gain  a working  knowledge  of  the  kindergarten  setting; 
teacher  views  on  technology  and  literacy;  learner  attitudes  toward  technology  and  areas  of  early  literacy  in  which  (a) 
materials  are  lacking;  and  (b)  a technology-based  solution  might  offer  added-value.  In  the  early  stages  of  a small 
scale  innovation,  researchers  sought  collaboration  with  schools  who  would  not  only  be  involved  in  the  initial 
analysis,  but  also  in  the  design  and  evaluation  of  the  learning  environment.  Two  local  schools  were  selected  to 
participate  in  the  needs  and  context  analysis,  based  on:  their  interest  in  the  project;  their  track  records  in  endeavors 
to  help  teachers  make  effective  use  of  technology;  and  their  commitment,  as  evidenced  by  the  willingness  to  invest 
their  own  time  and  resources.  Data  were  collected  through  observations  during  site  visits;  interviews  with  school 
leaders;  a questionnaire  administered  to  all  teachers;  and  a verbally-administered  questionnaire  for  kindergarten 
through  first  grade  learners.  At  the  same  time,  a literature  review  was  conducted  to  gain  insights  into  early  literacy; 
technology  for  young  children;  and  technology  for  early  literacy.  Finally,  document  analysis  was  conducted  on  the 
language  curricula  used  in  the  two  schools  as  well  as  the  national  interim  targets  for  early  literacy. 

Toward  generating  ideas  for  potential  solutions,  two  types  of  exploration  were  carried  out,  evolving  as 
insights  from  the  analysis  grew:  review  of  existing  products  (described  in  literature,  in  sales  brochures  or  at 
conferences  like  the  National  Educational  Computing  Conference);  and  contacts  were  sought  with  practitioners  and 
experts  that  could  serve  as  ‘critical  friends’  to  the  endeavour. 

For  detailed  findings  from  this  phase,  please  refer  to  McKenney  and  Voogt  (2005).  The  list  below 
summarizes  the  main  findings  from  the  first  phase: 

Kindergarten  setting:  Most  classrooms  serve  about  25  children  and  have  2-3  computers;  additional 
computers  are  available  in  the  hallways. 
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Teacher  views:  Teachers  are  generally  positive  about  the  use  of  ICT;  feel  comfortable  with  computers  and 
are  open  to  learning  new  educational  applications.  They  find  literacy  important  area,  but  say  that  they 
concentrate  most  on  reading;  comparatively  little  emphasis  is  placed  on  learning  to  write. 

Learner  attitudes:  Learners  are  generally  quite  positive  about  the  use  of  ICT.  At  school,  they  mostly  use  the 
computer  to  play  games  and  85%  say  they  use  computers  at  home,  also. 

Early  literacy:  Children  develop  early  literacy  through  positive  relationships;  print-rich  environments;  daily 
reading;  experiences  that  develop  phonemic  awareness;  play  involving  literary  props;  and  firsthand 
experiences  that  expand  vocabulary. 

Technology  and  young  children:  Developmentally  appropriate  uses  of  technology  with  young  children  are 
integrated  into  the  regular  learning  environment;  equitably  accessible  to  all  children;  and  should  promote 
appreciation  and  positive  social  values. 

Technology  for  early  literacy:  Most  existing  technologies  for  early  literacy  focus  on  technical  aspects  of 
reading  or  multimedia  storybooks. 

Phase  2:  Design  and  construction.  The  findings  from  the  analysis  and  exploration  phase  were  used  to  elicit 
guidelines  that  shaped  the  design  of  the  PictoPal  intervention.  Given  the  existence  of  technology  tools  for  reading; 
the  lack  of  emphasis  and  tools  for  writing;  and  the  national  interim  targets  related  to  writing,  the  primary  focus  was 
determined  to  be:  understanding  the  nature  of  written  language.  With  this  in  mind,  PictoPal  would  be  designed  to 
help  children  recognize  (a)  the  relationship  between  spoken  and  written  language;  and  (b)  purposeful  reading  and 
writing.  Inspired  by  a principal's  idea  to  use  pictograms  to  help  children  create  texts,  we  decided  to  create  a means 
through  which  children  could  ‘write'  and  print  documents  with  the  aid  of  the  computer.  The  printed  products 
would  then  be  used  either  as  literary  props  in  dramatic  play,  or  for  authentic  purposes.  We  viewed  this  use  of 
technology  as  a form  of  constructionism  (Papert,  1980),  where  children  “learn  by  making.” 

The  design  and  development  of  PictoPal  was  guided  by  several  sets  of  principles,  which  were  derived  from 
the  analysis  and  exploration  phase.  These  relate  to  tenets  underpinning  the  overall  endeavour  (core  convictions); 
parameters  for  the  product  (product  guidelines)  and  heuristics  for  developing  (process  guidelines),  which  are 
summarized  below: 

Core  convictions 

0 Children  want  to  express  themselves  in  print,  even  before  they  are  able  to  read; 

O The  powerful  combination  of  images  and  print  can  help  children  to  express  themselves; 

0 The  computer  can  offer  added  value  by  helping  children  to  focus  on  formulating  their  message 
rather  than  the  mechanics  of  writing. 

Product  guidelines 

0 PictoPal  should  be  an  open  learning  environment  that  facilitates  children's  active  role  in  creating 
written  products  which  may  be  used  for  a variety  of  purposes  and  in  authentic  ways. 

0 PictoPal  on-computer  activities  should  elicit  dialogue  and  collaboration,  preferably  with  adults  as 
well  as  peers. 

0 The  products  of  PictoPal  computer  activities  should  be  used  in  related  off-computer  activities,  to 
promote  print-enriched  play  and  authentic  use,  and  to  encourage  interaction  and  language  use  in 
meaningful  social  settings 
Process  guidelines 

0 The  design  of  PictoPal  should  be  conducted  in  cooperation  with  teachers  to  enhance  the 
practicality  of  the  design,  and  with  language  experts  to  enhance  the  legitimacy  of  the  design. 

0 The  design  of  the  PictoPal  computer  environment  should  be  informed  by  usability  testing  so  as  to 
address  the  ergonomic  needs  of  kindergarteners. 

Following  a developer  screening  of  a global  outline  and  then  a paper-based  prototype,  five  successive 
functional  prototypes  were  developed.  The  paper-based  prototype  and  first  functional  prototype  contained  both 
closed  assignments  and  semi-open  ones.  A major  change  between  the  first  and  second  working  prototypes  was  the 
shift  to  using  a commercial  software  package  (Clicker®)  to  deliver  the  PictoPal  activities.  This  decision  was  made 
because:  it  allowed  developers  to  focus  more  on  developing  the  learning  content  and  less  on  developing  the 
technology;  evaluation  of  the  first  working  prototype  showed  the  closed  activity  types  to  be  superfluous;  and  90%  of 
the  semi-open  activity  functions  were  present  in  the  Clicker®  program. 
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Phase  3:  Evaluation  and  reflection.  In  total,  seven  formative  evaluations  were  conducted  with  the  five  functional 
prototypes.  Across  the  evaluations,  the  following  aspects  were  studied:  pupil  learning  gains;  pupil  engagement  with 
PictoPal;  integration  of  on  and  off-computer  activities;  and  teacher  experiences  as  enactors  or  designers  of  PictoPal 
activities.  The  first  study  (A)  explored  the  feasibility  of  the  design  of  the  first  prototype.  The  second  prototype  was 
primarily  designed  by  one  teacher,  the  design  process  was  the  focus  of  study  (B)  and  the  learning  gains  were 
evaluated  in  the  third  study  (C).  The  third  prototype  was  used  for  two  studies,  on  learner  engagement,  one  focusing 
more  on  the  off-computer  activities  (D)  and  the  other  on  the  on-computer  interactions  (E).  The  sixth  study  (F), 
conducted  with  the  fourth  prototype,  focused  on  the  integration  of  on-  and  off  computer  activities,  and  correlation  to 
learning  gains.  Conducted  with  the  fifth  prototype,  seventh  study  to  date  (F)  examines  the  influence  of  teacher 
perceptions  of  technology  as  well  as  their  beliefs  about  teaching  and  learning  on  the  way  they  integrate  on-  and  off- 
computer  activities,  as  well  as  correlation  to  learning  gains.  As  the  function  of  this  paper  is  to  describe  the  overall 
design  research  approach,  and  because  each  of  these  studies  has  been  reported  in  detail  elsewhere  (Cviko, 
McKenney,  & Voogt,  2009;  McKenney  & Voogt,  2009a, , 2009b),  Table  1 presents  only  an  overview  of  the  seven 
studies,  their  respective  foci  and  methods  used. 


Study: 

A 

B 

C 

D 

E 

F 

G 

Prototype 

1 

2 

3 

4 

■ B 

5 

Focus 

pupil  learning  gains 

pupil  engagement 

integration 

teacher  experiences 

Methods 

Pre/post-test 

On-pc  observation 

Off-pc  observation 

Teacher  interviews 

Details 

McKenney  & Voogt,  2009a 

McKenney  & Voogt,  2009b 

Cviko,  McKenney  & Voogt  2009 

Table  1;  Overview  of  PictoPal  studies 


Findings  from  the  earlier  studies  informed  both  the  development  of  the  PictoPal  tool,  and  development  of 
the  flanking  research.  Study  A demonstrated  that  children  are  able  to  use  the  software,  and  enjoy  the  experience.  It 
showed  that  children  require  assistance  in  understanding  what  to  do  on  each  screen,  that  the  level  of  assistance 
needed  does  decrease  over  time,  but  that  some  supervision  remains  necessary.  It  also  showed  that  children  did  not 
require  the  closed  assignments  to  help  learn  the  pictogram  ‘vocabulary’  as  suspected,  because  the  images  were 
sufficiently  self-explanatory.  Finally,  it  showed  no  learning  gains;  the  researchers  hypothesized  that  this  was  due  to 
the  limited  duration  of  the  intervention  (4  sessions).  This  promoted  several  changes  in  the  learning  environment, 
which  were  incorporated  into  the  second  prototype:  closed  activities  were  eliminated;  the  number  of  sessions  was 
doubled;  and  a commercial  software  package  (Clicker®)  was  adopted.  In  addition  to  allowing  researchers  to  focus 
more  on  content  design  instead  of  software  design,  the  adoption  of  Clicker  ® enabled  the  possibility  of  involving 
teachers  directly  in  the  design  process.  This  was  viewed  as  desirable  because  teacher  involvement  in  design  could 
help  facilitate  the  implementation  of  PictoPal.  When  examined  in  Study  B,  it  became  clear  that  substantial 
pedagogical,  linguistic,  and  technological  expertise  would  have  to  be  provided  to  enable  teacher  design  of  PictoPal 
materials.  However,  the  learning  gains  (Study  C)  resulting  from  the  use  of  the  (heavily  scaffolded)  teacher-made 
materials  were  high  enough  to  warrant  further  exploration.  By  the  third  prototype,  the  basic  structure  of  PictoPal 
on-computer  materials  had  become  rather  stable,  and  more  attention  was  given  the  nature  of  pupil  engagement, 
while  creating  their  documents  on  the  computer  (Study  E)  and  while  ‘using’  their  products  in  the  classroom  (Study 
D).  While  developmentally  appropriate  practices  had  indicated  that  young  children’s  technology  use  should  be 
integrated  with  off-classroom  activities,  the  data  prompted  researchers  to  explore  a causal  link  between  the  level  of 
integration  and  pupil  learning  gain  (Study  F).  Initial  findings  indicate  that  this  causal  relationship  exists,  and  so 
factors  affecting  integration  (teacher  perceptions  and  beliefs)  were  also  taken  into  account  in  Study  G;  here  too,  the 
hypothesized  relationship  was  shown. 
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Outputs 

Final  Intervention.  As  PictoPal  research  is  still  in  full  swing,  a final  version  cannot  yet  be  described.  However,  the 
main  elements  of  the  fifth  prototype  are  described  here.  As  previously  mentioned,  children  use  PictoPal  materials 
designed  for  Clicker  ® software  to  create  and  print  out  documents,  which  are  then  used  in  off-computer  activities. 
Throughout  a PictoPal  module,  children  go  through  eight  sessions,  each  with  the  same  basic  elements:  (1) 

Classroom  starter  activity;  (2)  On-computer  text  production;  and  (3)  Off-computer  classroom  application.  The  eight 
on-computer  activities  scaffold  the  children’s  writing  of  different  types  of  texts  (list,  letter  story,  instruction,  etc.), 
the  complexity  of  which  increases  over  time  (e.g.  the  first  activity  could  be  a simple  list;  a middle  activity  could  be  a 
several  sentence-long  report;  a final  activity  could  be  a whole-page  story).  Figure  2 shows  how  one  of  the  sentences 
in  a weather  forecast  is  constructed.  Children  click  on  buttons  in  the  lower  half  of  the  screen,  and  the  corresponding 
words  are  typed  for  them  in  the  upper  half.  Figure  3 illustrates  the  associated  off-computer  classroom  activity: 
presenting  the  weather  forecast  to  the  class.  In  the  most  recent  study  (G),  teachers  were  supported  in  implementing 
PictoPal  through  an  introductory  workshop  and  a teacher  guide.  The  weather  forecast  page  from  the  teacher  guide  is 
shown  in  Figure  4. 
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Figure  2:  Weather  forecast  grid 


Figure  3:  Presenting  the  weather  forecast 


Las  3 - Waarteancht 

(q)  Eindfasultaat  van  deze  as 

In  deze  les  gaan  de  leeriingen 
een  weerbercht  maken  met 
behulp  van  Clicker©.  Aan  het  eind 
van  de  les  presenteren  de 
leeriingen  het  weerbericht  op  een 
(nep)  TV.  In  Figuur  4 staat  een 
albeelding  van  een  mogelijk 
eindresultaaL 

r S Klassanacbvitalt  vooraf 

In  deze  les  maken  de  leeriingen  kennis  met  het  weerbericht  Laat  een  voorbeeld 
van  een  weerbericht  zien.  Dit  kan  een  kort  filmpje  zijn  of  een  albeelding  uit  de 
krant.  Praat  met  de  leeriingen  over  het  weerbencht.  Waar  gaat  het  over7  Welke 
weersoorten  zijn  er?  Denk  aan  mooi/slecht  weer  en  warm/koud  weer.  maar  ook 
aan  sneeuw,  zon.  regen.  etc. 

— Computaractivrteit 

Deze  les  bestaat  uit  drie  grids:  De  leeriingen  beginnen  met  het  weerbericht  Ze 
moeten  een  keuze  maken  tussen  vandaag'  en  'morgen'  en  kiezen  wat  voor  weer 
het  is.  Hieronder  is  de  eerste  en  de  laatste  grid  afgebeeld. 


I - Klassanacbvitalt  achlaraf 

p-* 

Maak  van  een  doos  een  TV  en  laat  de  leeriingen  hierin  hun  urtgeprinte  weerbericht 
p resente ren/voortezen.  Zorg  eventueel  voor  accessoires  (paraplu.  zonnebril)  en/of 
kartonnen  kaarten  met  symbolen  (zon.  wolk,  regen)  die  de  leeriingen  kunnen  laten 
zien  tijdens  de  TV  show’. 

Figure  4:  The  weather  forecast  lesson  in  the 
teacher  guide 


vandaag  is  het  warm 
morgen  gaat  de  wind  hard  waaien 

S3 

de  lueht  wordt  donker  door  de  wolken. 


Figuur  4 - Het  weerbericht  voor  morgen 


Design  principles.  In  light  of  our  research  findings  and  experiences,  we  revisit  our  initial  product  and  process 
guidelines,  and  comment  on  what  we  have  learned  through  the  PictoPal  case. 

Product  guidelines 

0 Open  learning  environment 

■ A semi-open  environment  is  more  practical  for  the  on-computer  activities  than  a fully 
open  one  because:  (a)  existing  software  can  be  used;  and  (b)  for  this  age  range,  the 
amount  of  flexibility  in  semi-open  assignments  seems  sufficient;  and  (c)  the  level  of 
guidance  necessary  for  a fully  open  environment  is  likely  to  be  highly  impractical. 

■ While  fully  open,  the  environments  for  off-computer  activities  must  be  explicitly  (not 
casually)  linked  to  the  type  of  product  generated  through  the  on-computer  activity. 

O Activities  should  elicit  dialogue  and  collaboration 

■ During  the  first  few  sessions,  PictoPal  discussions  during  its  use  generally  center  around 
operating  the  computer.  By  devoting  time  to  computer  operations  before  starting  the 
PictoPal  sessions  (e.g,  one  full  pre-session  module  in  studies  F and  G),  more  on-task 
behavior  can  be  elicited. 

■ Children  discuss  PictoPal  with  each  other  and  with  adults,  although  their  conversations 
are  more  on-task  with  adults.  However,  adult  guidance  is  very  impractical  for  most 
schools.  Student  teachers  and  parent  volunteers  have  been  successful,  but  are  difficult  to 
rely  on;  tutor  systems  involving  6th  graders  have  also  been  successful. 

O PictoPal  products  should  be  used  in  related  off-computer  activities 

■ Single  page  products  work  better  than  multiple  pages  ones,  because  they  are  easier  for  the 
children  to  manipulate. 

■ When  children  use  the  computer  in  pairs,  their  classroom  applications  should  be  designed 
around  continued  partnership. 
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Process  guidelines 

0 PictoPal  should  be  designed  together  with  teachers  and  language  experts 

■ Teachers  can  clearly  bring  realistic,  implementable  ideas  to  the  drafting  table.  They  help 
ensure  that  the  classroom  activities  are  within  the  zone  of  proximal  development  for  most 
other  teachers.  Product  development  is  greatly  enhanced  when  this  kind  of  input  is 
sought  early  on. 

■ Measures  should  be  taken  to  mitigate  a sense  of  feeling  overwhelmed.  Even  highly 
motivated  teachers  can  have  become  more  focused  on  ‘getting  the  job  done’  and  not  see 
when  they  are  making  poor  design  decisions. 

■ Providing  sample  materials  and  completed  examples  to  teachers  early  on  is  crucial  to 
teacher  understanding 

■ Pupil  learning  gains  are  highly  motivating:  Teachers  are  eager  to  continue/conduct 
PictoPal  work  when  they  see  positive  results 

■ Substantial  technical,  process  and  conceptual  support  must  also  be  provided  to  teachers 
designing  PictoPal  materials. 

0 PictoPal  design  should  be  informed  by  usability  testing 

■ The  misguided  temptation  to  include  fun  and  interesting,  yet  often  distracting  features  on 
screen  must  be  resisted. 

■ On-  and  off-computer  activities  must  be  designed  simultaneously  or  cyclically  (not  one 
after  the  other)  to  optimize  integration. 

Interaction  with  practice 

Implementation.  The  PictoPal  project  has  been  developing  at  a relatively  small  scale.  Across  the  seven  studies,  four 
schools  have  been  involved.  Implementation  strategies  are  considered  from  the  start  of  collaboration  with  a school. 
For  example,  efforts  are  made  to  accommodate  teacher  calendars,  to  draw  links  with  the  existing  language 
curriculum,  and  to  address  topics  recommended  by  teachers.  Intensive  support  (coaching,  feedback)  has  been 
provided  when  teachers  design  the  on-computer  activities.  Far  less  support  has  been  given  when  teachers  use  ready- 
made activities  and  only  refine  the  classroom  applications.  Where  teachers  have  taken  the  enactor  role,  the  PictoPal 
has  been  implemented  through  an  orientation  workshop  and  a guidebook.  As  with  many  interventions,  this  has 
worked  best  when  one  person  assumes  accountability  for  the  implementation;  usually  either  the  language 
coordinator  or  the  early  grades  coordinator.  During  the  formative  evaluations  of  PictoPal  prototypes,  various 
approaches  to  offering  pupil  guidance  have  been  tried  out.  Student  teachers,  parent  volunteers  and  sixth  grade  tutors 
have  all  been  conducted,  and  met  with  some  success.  However,  teachers  find  these  solutions  to  require  too  much 
time  and  hassle  to  organize  on  a regular  basis.  Participating  teachers  have,  repeatedly,  commented  that  they  learn 
from  the  PictoPal  experience.  Through  future  studies,  we  would  like  to  better  understand  the  nature  of  this  learning 
and  how,  if  at  all,  it  can  be  further  supported. 

Diffusion.  The  PictoPal  endeavor  was  started  as  an  exploratory  research  and  development  project,  funded  by  the 
National  Institute  for  Curriculum  Development.  It  was  not  intended  for  large  scale  use  and  uptake.  However,  after 
years  of  iterative  refinement,  the  PictoPal  intervention  may  be  nearing  a degree  of  maturity  that  warrants 
consideration  of  how  to  promote  diffusion  of  the  intervention.  To  date,  diffusion  of  the  research  and  its  outputs  have 
taken  place  through  researcher  conferences;  practitioner  workshops;  and  journal  articles. 

Discussion 


Reflections  on  the  PictoPal  initiative 

A design  research  approach  was  used  to  gain  insight  into  desirable  program  characteristics,  implementation 
strategies  and  the  role  of  teachers  as  designers  of  curriculum  materials.  Following  a context  analysis  (McKenney  & 
Voogt,  2005),  literature  search  and  curriculum  analysis,  multiple  prototypes  were  (re)designed  and  formatively 
evaluated  (Cviko,  McKenney,  & Voogt,  2009;  McKenney  & Voogt,  2009a,  2009b;  Voogt  & McKenney,  2007). 
The  approach  to  used  evidences  the  characteristics  described  in  literature.  These  are  briefly  recapitulated  below, 
using  the  set  offered  by  Reinking  and  Bradley  (2008): 

Intervention-centered:  Having  a positive  impact  on  early  literacy  learning  is  central  to  the  initiative. 
Theoretical:  PictoPal  development  is  informed  by  research  findings  and  theoretical  works;  it  contributes  to 
theory  building  about  emergent  literacy  and  technology  integration. 

Goal-oriented:  Aims  to  explore  how  technology  can  offer  added  value  in  literacy  development. 
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Adaptive  and  iterative:  The  tools  and  supports  have  evolved  in  light  of  the  experiences  and  research 
findings. 

Transformative:  The  intervention  stimulates  new  practices  in  kindergarten  classrooms. 

Methodologically  inclusive  and  flexible:  Across  the  cycles,  qualitative  and  quantitative  data  were  collected; 
data  source  decisions  were  influenced  by  contextual  opportunities  and  constraints. 

Pragmatic:  Research,  development  and  implementation  efforts  were  driven  by  the  desire  to  achieve  an 
effective,  sustainable,  scalable  intervention. 

The  PictoPal  endeavor  has  yielded  a functional  intervention  which  has  demonstrated  its  potential  to 
positively  influence  kindergarten  children’s  understanding  of  the  functions  of  written  language.  Further,  the 
successive  studies  have  shown  that  learner  gains  increase  along  with  teacher  involvement  in  the  design  of  activities. 
We  think  this  is  because  the  experience  helps  teachers  to  better  integrate  the  on-computer  activities  into  the  ongoing 
classroom  practices  in  general,  and  the  PictoPal  off-computer  activities  in  particular.  We  also  conclude  that  the 
classroom  integration  is  affected  by  teacher  perceptions  of  technology  and  their  own  beliefs  about  teaching  and 
learning. 

Lessons  learned  for  other  design  research  initiatives 

This  paper  concludes  by  discussing  several  lessons  learned  from  the  PictoPal  studies,  which  may  be  useful 
for  other  design  research  endeavors.  These  pertain  to  sustaining  design  and  development;  tensions  and  tradeoffs 
when  what  is  theoretically  legitimate  is  impractical;  and  methodological  challenges. 

Sustaining  design  and  development.  The  PictoPal  project  was  initiated  as  an  exploratory  research  and  development 
endeavor;  and  funding  was  only  available  for  the  first  year  (development  of  the  initial  working  prototype). 
Thereafter,  seven  ( undergraduate  students  continued  the  work  in  the  form  of  their  capstone  assignments.  Planning 
for  capstone-sized  chunks  primarily  during  springtime  (the  semester  when  students  are  usually  available),  and 
tailoring  the  details  of  the  studies  to  accommodate  findings  from  previous  studies  as  well  as  (where  feasible) 
participating  teacher  and  student  preferences,  allowed  this  initiative  to  mature.  The  benefit  of  experience  and 
publications  rooted  in  the  student  work  has  been  demonstrated  through  two  recently-awarded  grants,  each  for  a 4- 
year  term.  This  approach  has  been  a viable  way  to  address  the  chicken-and-egg  kind  of  problem  that  many  (design) 
researchers  face:  We  have  to  know  enough  about  something  to  be  able  to  secure  funding  for  it;  but  without  the 
funding  for  research  - how  do  we  learn  enough? 

Tensions  and  tradeoffs.  From  early  on  in  the  project,  it  became  clear  that  adult  guidance  during  on-computer 
activities  was  ideal  because  it  (a)  was  developmentally  appropriate;  and  (b)  offered  the  greatest  potential  for  on-task 
dialogue.  However,  such  an  approach  is  rarely  feasible  in  kindergarten  classrooms.  The  time  and  hassle  involved  in 
organizing  adults  or  even  older  children  as  tutors  is  too  demanding  for  most  teachers.  In  the  PictoPal  studies, 
teachers  have  been  willing  to  experiment  with  varying  approaches  to  addressing  this  tension.  But  they  generally 
indicate  that  failure  to  satisfactorily  address  issue  is  currently  the  only  substantial  weakness  in  the  intervention.  But 
that  weakness  is  so  great  that,  even  when  the  values  and  benefits  of  older  guides  has  been  clarified,  teachers  question 
the  viability  of  PictoPal.  Practicality,  it  would  seem,  supersedes  legitimacy,  in  this  case.  It  is  conjectured  that  this  is 
no  isolated  case,  and  other  design  research  endeavors  would  do  well  to  embrace  this  when  unbridled  design  ideals 
threaten  to  take  charge. 

Methodological  challenges.  In  design  research,  more  than  in  other  research  approaches,  the  tendency  for  researchers 
to  take  on  additional  roles  (e.g.  facilitator,  designer)  presents  a significant  challenge  to  the  rigor  of  a study.  In  the 
PictoPal  work,  several  tactics  have  been  employed  to  mitigate  those  challenges.  First,  we  strive  to  provide  an 
explicit  conceptual  framework  for  each  PictoPal  study,  so  that  readers  can  make  their  own  analytic  generalizations. 
We  are  still  learning  how  to  characterize  salient  elements  of  the  context  to  further  assist  in  this.  Second,  we  rely  on 
triangulation  of  data  collection  methods  and  sources.  Third,  while  our  student  researchers  have  been  subject  to  some 
biases,  by  collaborating  on  their  mentors’  projects,  we  do  consider  them  more  objective  than  those  who  have 
actually  designed  PictoPal  materials.  Finally,  we  try  to  employ  both  inductive  and  deductive  data  analysis.  That  is, 
we  not  only  classify  according  to  schemes  based  on  our  existing  understanding;  but  we  also  explore  data,  trying  to 
make  sense  of  it  and  allowing  themes  to  ‘bubble  up’  to  the  surface. 

In  closing,  we  return  to  the  three  main  motives  for  conducting  design  research:  increasing  the  relevance  of 
research;  developing  empirically-grounded  theories;  and  increasing  the  robustness  of  design  practice.  The  PictoPal 
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endeavour  has  yielded  a practical  solution  to  the  problems  of  (a)  meeting  early  literacy  standards  for  the  functions  of 
reading  and  writing;  and  (b)  struggling  to  integrate  technology  in  developmentally  appropriate  fashion.  Data  show 
that  children  learn  from  using  PictoPal,  and  anecdotal  evidence  suggests  that  teachers  learn  from  the  experience  as 
well.  The  findings  contribute  to  implementation  theory  by  demonstrating  the  correlation  between  teacher  beliefs 
and  technology  integration;  and  the  correlation  between  the  level  of  technology  integration  and  learning  gains.  They 
also  substantiate  our  convictions  that,  in  a day  and  age  where  every  additional  hour  spent  at  the  computer  means  that 
much  less  time  spent  role-playing,  building  or  painting,  technology  is  only  worth  children’s  time  and  teachers’  effort 
when  carefully  designed  and  implemented.  Finally,  we  hope  that  some  of  our  design  process  guidelines  and  design 
product  guidelines  will  help  inform  future  initiatives  to  design  technology  for  early  literacy. 
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Enhancing  user  participation  in  an  art  gallery  space  through  the 
development  of  a user  generated  wiki 
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Abstract 

A problem  exists  today  around  art  galleries.  An  galleries  that  do  not  keep  permanent  and  publicly 
accessible  collections  or  records  of  the  artwork  they  display  do  not  provide  opportunities  for  artists  and  viewers  to 
revisit  artwork  that  has  been  removed  from  the  gallery.  The  purpose  of  this  study  is  to  test  the  usability  of  a wiki 
platform  for  a university  art  gallery  that  allows  past  and  present  exhibiting  student-artists  and  any  other  interested 
parties  to  create  professional  profiles  and  alternative  exhibition  spaces  on-line  in  order  to  engage  with,  deepen, 
reflect  upon,  share  among,  translate  between  and  expand  their  experiences  with  and  understandings  of  art. 

Introduction 

The  use  of  the  Internet  by  people  of  all  ages  around  the  world  (Lenhart  2009;  Sutter  2009)  to  create,  remix, 
and  share  media  content  (Lenhart  & Madden,  2005)  through  engagement  in  “participatory  cultures”  (Jenkins,  2006) 
is  rapidly  increasing  (Zuckerberg,  2009).  According  to  Lu  (2008),  teaching  professionals  in  the  21st  century  “need  to 
think  about  how  to  engage  and  motivate  students  into  using  their  favorite  and  familiar  digital  media  as  effective 
learning  technologies”  (p.  48). 

At  the  Pennsylvania  State  University  (2007),  over  forty-thousand  students  are  currently  enrolled  at  the 
University  Park  campus,  just  one  branch  of  the  larger  University  system.  According  to  a survey  conducted  by  the 
Educational  Technology  Services  at  Penn  State,  83%  of  Penn  State  students  are  in  the  Facebook  social  network, 

93%  own  cell  phones,  and  nearly  90%  own  MP3  players  (Camplese,  2008).  The  every-day  lives  of  these  college 
students  are  becoming  increasingly  interwoven  with  the  use  of  contemporary  popular  technologies. 

As  a teaching  and  learning  space  at  Penn  State,  the  Edwin  W.  Zoller  Gallery  is,  centrally,  a space  for 
students  to  push  the  boundaries  of  how  they  create  and  think  about  art.  Each  academic  semester,  the  Zoller  Gallery 
hosts  as  many  as  seven  public  exhibitions  of  student  art  work,  and  it  is  expected  that  entire  classes  of  students  will 
come  into  the  gallery  with  their  instructors  at  any  given  time  to  talk  about  the  artwork  on  display.  However,  because 
the  Zoller  Gallery  keeps  no  permanent  collection  of  the  artwork  it  displays,  unlike  a museum,  students  have  no 
means  to  revisit  an  exhibition  of  artwork  at  the  gallery  after  one  has  been  removed  to  make  way  for  the  next. 

Finding  this  inability  of  students  to  revisit  prior  learning  experiences  problematic,  this  design-based 
research  study  seeks  to  provide  contextually  relevant  and  meaningful  opportunities  for  more  media-rich  and 
dynamically-connected  art  educational  experiences  within  and  beyond  the  Zoller  Gallery.  By  providing  ways  for 
students  to  engage  with  social  media,  both  inside  and  outside  of  the  gallery,  about  the  art,  artists,  and  events 
happening  there,  the  gallery  becomes  more  culturally  responsive  and  connected  to  contemporary  student  culture. 
Through  the  adaptation  of  social  media  to  enhance  and  extend  the  opportunities  for  teaching  and  learning  about  and 
through  art,  the  Zoller  Gallery  can  offer  students  an  alternative  to  the  limitations  of  their  current  gallery  experiences 
by  providing  them  with  an  emergent  multi-media  exhibition  and  conversation  platform  to  germinate. 

Study 

The  purpose  of  this  study  is  to  test  the  usability  of  a wiki  platform  associated  with,  but  not  bound  to  or 
restricted  by,  the  Zoller  Gallery.  This  wiki  allows  past  and  present  exhibiting  student-artists  and  any  other  interested 
parties  to  create  personal  (artist)  profiles  and  alternative  exhibition  spaces  on-line  in  order  to  engage  with,  deepen, 
reflect  upon,  share  among,  translate  between  and  expand  their  experiences  with  and  understandings  of 
art.  Following  the  creation  of  an  artist  profile,  artists  upload  an  artwork,  provide  descriptions  of  the  work,  and  allow 
other  users  of  the  wiki  platform  to  provide  comments  and  feedback  on  a whiteboard  associated  with  that 
artwork.  Figure  1 provides  a screenshot  of  this  discussion  and  feedback  feature  of  the  wiki  platform. 
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Figure  1.  Art  gallery  wiki  discuss  page.  This  figure  provides  an  example  of  the  whiteboard  commenting  feature  for 
the  artwork  entitled  Catching  a Bus  at  PSU. 

A qualitative  usability  test  method  was  used  to  conduct  this  study.  We  followed  Goto  and  Cotier’s  four- 
step  process  in  setting  up  our  usability  test.  Goto  and  Cotier  recommend  planning  and  prepping,  finding  participants, 
conducting  the  study,  and  finally,  analyzing  data  and  making  recommendations  based  on  the  results  (2005).  We 
followed  their  guidelines  and  used  their  Likert-type  post  session  survey  items  as  part  of  our  user  post-test  survey. 

The  goals  of  our  usability  test  were  to  provide  specific  feedback  on  common  tasks  within  the  system  and  determine 
expected  performance  on  the  current  site  and  identify  serious  problems  prior  to  the  next  phase  of  production.  By 
conducting  a usability  test,  we  wanted  to  answer  the  following  questions:  Are  the  basic  tasks  intuitive  for  a new  user? 
Are  the  menu  options  and  features  intuitive  for  a new  user? 

Method 

Our  study  consisted  of  four  graduate  students  that  regularly  engage  and  participate  in  an  art  gallery 
space.  We  gathered  two  types  of  background  information.  First,  we  asked  how  many  hours  a participant  spent 
online  per  day,  and  second,  we  asked  our  participants  to  complete  a pre-test  power  user  scale  survey  to  determine 
how  “tech-savvy”  a person  is.  Marathe  et  al.  created  the  power  user  scale  for  classification  purposes  when 
conducting  media  effect  studies  (2007).  The  power  user  scale  consists  of  nine  Likert-type  items  and  are  provided  in 
Appendix  A. 

Each  session  lasted  approximately  30  minutes.  During  the  usability  test,  each  subject  was  greeted  and  then 
oriented  to  the  design  by  a test  monitor.  Subjects  were  given  four  tasks  to  complete  while  the  test  monitors 
observed.  Participants  were  asked  to  think-aloud,  i.e.  talk  out  their  thought  process,  while  they  completed  the  four 
tasks.  The  four  tasks  are  listed  in  Appendix  B.  Following  the  tasks,  a post-test  survey  was  given  in  addition  to  a 
debriefing  on  the  significance  of  the  study. 


Results 

The  number  of  hours  spent  online  was  roughly  equivalent  across  subjects,  showing  that  our  participants  did 
have  experience  in  online  web  applications  with  an  average  of  5.72  hours  spent  online  per  day.  High  power  users 
consisted  of  participants  who  scored  high  across  all  items  on  the  power  user  scale  and  low  power  users  scored 
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low.  We  found  that  two  of  our  participants  scored  high  (average  of  8.278)  and  two  of  our  participants  scored  low 
(average  of  4.61 1).  In  our  study,  the  power  user  scale  items  were  compared  using  a Multivariate  factor  analysis  to 
determine  that  the  scale  was  reliable  (Cronbach’s  alpha  of  0.8033). 

When  comparing  our  high  and  low  power  users  across  post-test  survey  results,  we  found  that  the  high 
power  (tech-savvy)  users  consistently  scored  higher  across  all  items.  These  items  are  listed  in  Table  1.  Each  item  is 
a score  out  of  5 (l=strong  disagree;  5=strongly  agree). 


Low  Power  Users 

High  Power  Users 

Easy  to  complete  tasks 

4 

5 

Clear  Navigation 

4.5 

5 

Pleasing  Look  and  Feel 

3.5 

5 

Relevance  of  images  to  content 

4 

5 

Relevance  of  content  (text) 

4 

5 

Easy  to  use 

4 

5 

Clear  page  layout 

4 

5 

Inviting  to  use 

4 

4.5 

Clear  labeling  of  links 

4.5 

4.5 

Recommend  a friend  to  use 

3.5 

5 

Table  1.  Low  vs.  High  Power  Users.  This  table  provides  the  score  for  each  group  of  participants  on  the  post-test 
survey. 


Additionally,  we  provided  three  post-test  survey  open-ended  questions.  The  participants  that  we  classified 
as  low  power  users  found  that  the  best  features  of  the  wiki  platform  were  the  numerous  ways  to  post  and  transmit 
ideas.  The  high  power  users  thought  the  best  feature  of  the  wiki  platform  was  the  ease  of  use.  The  low  power  users 
found  that  there  was  a lack  of  clarity  with  certain  tasks  (e.g.  the  discussion  board)  while  the  high  power  users  were 
pleased  with  the  system  and  listed  the  generic  color  scheme  as  the  only  complaint.  When  asked  what  would  you  do 
to  enhance  the  site,  one  low  power  user  participant  called  for  more  simplification  of  the  features  in  the  wiki 
platform.  One  of  the  high  power  user  participants  asked  for  a tagging  system  to  categorize  the  types/mediums  of 
artwork. 

During  the  think-aloud  process,  subjects  pointed  out  a few  important  concerns  that  were  not  addressed  in 
our  post-survey  results.  Below  we  provide  what  we  found  to  be  the  most  important  concerns: 

> Lack  of  information  on  the  goals  and  outcomes  of  the  site 

> Lack  of  an  “update”  button  after  editing  text  (current  system  automatically  updates) 

> Problem  finding  the  login  area/information  (current  system  has  login  on  top  right) 

> Majority  of  feedback  will  be  text-based,  so  make  the  whiteboard  discussion  an  advanced  feature 

> Aesthetics  of  the  site  should  be  changed;  too  much  of  a template 

> Possibility  to  use  system  with  another  school  to  collaborate  and  comment  on  other  University 
student/artists’  artwork 

> Lack  of  ability  to  see  text  descriptions  when  you  hover  over  buttons 

> Lack  of  prompts  of  what  to  write  in  description  boxes 

> Explanation  needed  for  whiteboard  discussion  board 

> Lack  of  ability  to  make  artwork  full-screen 

Discussion 
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Our  results  show  that  the  participants  that  are  considered  high  power  users  and  “tech-savvy”  found  the  site 
to  be  easy,  clear,  inviting,  and  relevant.  In  other  words,  the  wiki  platform  architecture  is  appropriate  and  the  system 
serves  the  function  for  which  it  was  created.  This  is  an  important  finding  as  we  move  forward  to  our  next 
prototype.  For  those  participants  that  are  considered  less  “tech-savvy”,  the  findings  are  still  encouraging.  These 
participants  also  agreed,  although  not  as  strongly,  with  the  ease  and  clarity  of  the  site.  These  results  provide  us  with 
a positive  response  to  our  questions  regarding  the  intuitive  nature  of  the  tasks  and  menu  options. 

The  open-ended  questions  and  think-aloud  responses  provided  us  with  very  valuable  feedback  and 
suggestions  for  improvement.  A recurring  theme  across  participants  was  the  need  for  a better  aesthetic  and  color 
scheme.  The  suggestion  to  add  an  artistic  background  is  something  that  our  design  team  will  factor  into  our  next 
prototype.  We  did  get  a positive  response  with  the  simplicity  of  the  wiki  platform  and  therefore,  we  are  going  to 
ensure  that  adding  artistic  elements  do  not  interfere  with  the  simplicity  of  the  function  and  navigability. 

An  interesting  find  was  the  participant  confusion  and  realization  that  text  edits  were  automatically  synced 
across  the  system.  Our  participants  are  used  to  having  submit  buttons  upon  completing  a task  and  thus,  thought  that 
this  feature  was  needed.  This  finding  is  intriguing  because  it  suggests  that  we,  as  consumers  of  web  media,  are 
trained  by  web  designers  to  locate  buttons  when  editing  text.  In  order  to  accommodate  this  disconnect,  our  design 
team  will  add  a text  rollover  effect  that  informs  the  user  of  “editing  text”  and/or  “text  submitted"  during  the  edit 
process,  thus,  eliminating  the  need  for  a button. 

A non-design  suggestion  from  the  think-aloud  responses  provided  us  with  an  opportunity  to  reflect  on  our 
goals  and  outcomes  and  realize  the  potential  and  excitement  that  this  system  could  generate.  The  suggestion  was 
using  the  wiki  platform  with  another  school  to  collaborate  and  comment  on  other  University  student/artists’  artwork. 
Although  we  are  a few  prototypes  away  from  implementing  this  platform  on  that  scale,  this  suggestion  helps  us 
refocus  our  design  principles  and  rethink  our  social  problem.  By  increasing  collaboration  among  Universities,  our 
wiki  platform  could  serve  as  a solution  to  the  problem  of  removed  art  education;  removed,  as  in  removed  from 
society. 


Implications 

In  the  next  phase  of  our  work,  we  will  add  in  many  of  the  suggestions  from  this  study.  We  plan  to  move 
forward  with  the  prototype  and  increase  the  functionality  of  the  wiki,  including  the  amount  of  tasks  that  a user  can 
perform  without  sacrificing  its  simplicity,  clarity,  and  ease  of  use.  Through  this  and  subsequent  usability  studies,  we 
hope  to  create  a communication  platform  that  is  both  highly  usable  by,  and  useful  for,  student-artists  who  want  to 
engage  in  enhanced  and  extended  educational  experiences  around  artwork  exhibited  at  the  Zoller  Gallery. 
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APPENDIX  A 
Power  user  scale 

On  a scale  of  1 to  10  ( 1=LEAST  AGREE;  10=MOST  AGREE),  do  you: 

1 . Think  most  technological  gadgets  are  complicated  to  use. 

2.  Love  exploring  all  the  features  that  any  technological  gadget  has  to  offer. 

3.  Prefer  to  ask  friends  how  to  use  any  new  technological  gadget  instead  of  trying  to  figure  it  out 
myself. 

4.  In  interfaces  that  you  are  familiar  with,  do  you  get  frustrated  each  time  you  have  to  go  through 
basic  steps  designed  for  new  users. 

5.  Like  to  challenge  yourself  in  figuring  out  how  to  use  any  new  technology. 

6.  A little  bit  of  intuition  is  all  that  is  needed  to  figure  out  how  to  use  any  new  technology. 

7.  Many  of  your  friends  come  to  you  to  get  help  related  to  technological  gadgets. 

8.  Find  yourself  using  keyboard  shortcuts  on  the  computer. 

9.  Make  use  of  the  ‘Help'  feature  available  in  any  software. 

APPENDIX  B 
Task  Information 

1 . Your  first  task  is  to  enter  a description  for  your  artwork  Catching  a Bus  at  PSU.  Update  the  description 
with  a few  phrases.  If  you  are  prompted  to  log  in,  use  the  username  mike  and  password  mike.  Once  you 
have  accomplished  this  task,  briefly  discuss  the  positive  and  negative  aspects  of  the  system  during  this 
task. 

2.  Your  second  task  is  to  change  the  Artist  Biography  Information  for  your  video  by  going  to  My 
Profile.  Once  you  add  a few  phrases,  go  back  to  the  initial  artwork  page  ART  - ABOUT  tabs  and 
comment  on  what  is  different.  Briefly  discuss  the  positive  and  negative  aspects  of  the  system  during 
this  task. 

3.  Your  third  task  is  to  provide  feedback  for  the  artwork  on  the  ART  - discuss  tab.  If  you  haven't  already, 
you  may  watch  the  video  to  help  with  this  process.  Explore  the  function  of  the  discussion  board  and  see 
what  you  can  do.  Comment  on  your  experiences  including  suggestions  for  improvement. 

4.  Your  final  task  is  to  add  a question  to  the  Q+A  discussion  board.  Your  question  can  be  about  the  art 
video  or  a general  question.  Once  you  have  accomplished  this  task,  briefly  discuss  the  positive  and 
negative  aspects  of  the  system  during  this  task. 
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SimTIE-Math:  A Simulation-Game  on  Technology  Integration  for 

Mathematics  Learning 

Rodney  D.  Myers  and  Theodore  W.  Frick 

Descriptors:  simulation  games,  teacher  education 

Abstract 

This  article  describes  the  application  of  instructional  design  theories  and  game  design  principles  in  the  creation  of  a 
paper  prototype  for  a simulation  game  on  technology  integration  in  education  (SimTIE-Math).  The  study  employs  a 
design  and  development  research  design  in  which  artifacts  that  resulted  from  that  design  process  are  examined, 
including  design  documents,  iterations  of  game  components  and  rules,  e-mail  exchanged  by  the  designers,  notes 
from  meetings,  reports  from  playtesting  sessions,  and  other  related  documents.  The  purpose  of  the  study  is  to 
formulate  principles  for  the  use  of  instructional  design  theory  in  game  design.  By  exposing  and  examining  the 
design  process,  we  intend  to  contribute  to  design  knowledge  in  instructional  systems  design. 

This  article  describes  the  application  of  instructional  design  theories  and  game  design  principles  in  the 
design  and  development  of  a paper  prototype  for  a simulation  game  on  technology  integration  in  education 
(SimTIE-Math).  We  begin  with  a brief  explanation  of  design  and  development  research,  the  approach  used  in  this 
study.  Next  we  describe  the  purpose  of  this  study  and  the  methodology.  We  then  describe  the  iterative  design 
process  of  paper  prototyping  and  playtesting  and  how  the  findings  influenced  revisions  of  the  game  components  and 
rules.  Finally,  we  reflect  on  lessons  learned  from  the  experience  and  formulate  design  principles  for  the  use  of 
instructional  design  theory  in  game  design. 

Design  and  Development  Research 

Richey  and  Klein  (2007)  define  design  and  development  research  as  “the  systematic  study  of  design, 
development  and  evaluation  processes  with  the  aim  of  establishing  an  empirical  basis  for  the  creation  of 
instructional  and  non-instmctional  products  and  tools  and  new  or  enhanced  models  that  govern  their  development” 
(p.  1).  This  approach  has  a pragmatic  orientation  in  that  it  seeks  to  improve  practice  through  evidence -based  claims 
about  the  design  and  development  of  instructional  products.  The  study  of  the  design  process  and  its  artifacts  can 
elucidate  the  ways  in  which  designers  apply  theories,  principles,  and  methods  to  attain  satisfactory  results  in 
naturalistic  settings.  This  design  knowledge  may  be  characterized  by  “common  examples,  patterns,  and  principles, 
and  by  the  expertise  required  to  apply  these  generalities  in  specific  settings”  (Design-Based  Research  Collective, 
2003,  p.  9).  At  the  same  time,  it  can  also  contribute  to  theories  of  teaching  and  learning,  although  generalizability 
can  be  problematic  because  context  is  often  a significant  factor  in  the  enactment  of  an  intervention  (Hoadley,  2004). 

Purpose  of  the  Study 

One  of  the  major  challenges  to  teacher  education  is  to  provide  pre-service  teachers  with  enough  practice  in 
teaching  before  they  graduate.  It  is  costly  to  place  them  in  real  classrooms,  and  if  they  make  mistakes  there  are  real 
consequences  to  the  students  they  teach  dining  practica  and  student  teaching.  Simulations  are  routinely  used  in  a 
variety  of  professions  (e.g.  business,  healthcare,  engineering,  military,  law  enforcement  and  fire  fighting,  etc.)  to 
prepare  people  for  real-life  situations  that  they  are  likely  to  encounter.  SimTIE  is  intended  to  provide  pre-service 
teachers  with  opportunities  to  make  planning  decisions  for  selecting  student  learning  activities,  and  then  to 
experience  the  consequences  of  those  choices.  To  succeed  in  the  simulation,  teachers  will  need  to  give  up  the  notion 
of  "sage  on  the  stage"  and  adopt  a "guide  on  the  side"  perspective.  Moreover,  teacher  or  student  choice  of  learning 
activities  that  integrate  information  technology  will  be  necessary  in  order  to  provide  individualized  learning 
experiences  for  simulated  students  to  work  at  their  own  pace  on  activities  suited  to  their  skills  and  knowledge. 

The  purpose  of  this  study  is  to  examine  the  design  process  and  decisions  made  during  the  development  of  a 
paper  prototype  for  a simulation  game  on  technology  integration  in  education.  In  particular,  the  focus  is  on  the  ways 
in  which  instructional  design  theory  informed  the  decisions  regarding  simulation,  game,  and  pedagogical  elements 
(Aldrich,  2005).  We  have  sought  to  answer  the  following  design  and  development  research  questions.  First,  how  can 
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instructional  design  theories  be  applied  in  the  game  design  process?  Second,  what  design  principles  may  be 
formulated  from  a retrospective  analysis  of  the  design  and  development  process  and  artifacts? 

Methods 


Participants 

The  design  and  development  of  the  paper  prototype  was  undertaken  in  an  advanced  design  class  within  the 
context  of  an  Instructional  Systems  Technology  (1ST)  graduate  program  at  a large  midwestern  university.  The  class 
consisted  of  five  residential  students  and  one  distance  student.  Four  of  the  students  were  nearing  completion  of  their 
Master’s  degrees,  while  the  other  two  (including  the  first  author)  were  beginning  their  doctoral  studies.  The  second 
author  was  the  instructor  of  the  class  and  also  played  the  role  of  the  client  who  was  contracting  the  team  to  design 
the  instructional  product. 

The  second  author  had  previously  led  a team  of  students  in  adapting  a board  game,  the  Diffusion 
Simulation  Game  (DSG),  for  online  use  (Frick,  Kim,  Ludwig,  & Huang,  2003).  A few  of  the  students  in  the  current 
study  had  some  limited  experience  in  designing  board  games,  card  games,  puzzles,  and  digital  games.  To  prepare  for 
the  project,  the  students  created  and  tested  some  simple  games  from  Thiagarajan’s  (2003)  book  on  game  design. 
Thiagarajan  also  came  to  speak  with  the  class  about  game  design. 

Procedure 

Klabbers  (2006)  has  argued  that  the  analytical  sciences  employ  a variable  approach  (based  on  variables  and 
the  correlations  among  them)  as  a mode  of  explanation  while  the  design  sciences  should  employ  a process  approach 
(based  on  events  and  the  processes  that  connect  them,  cf.  Maxwell,  2004)  because  causality  is  purposefully  designed 
into  the  utilization  of  artifacts  with  the  ultimate  goal  of  changing  existing  situations  into  preferred  ones  (cf.  Simon, 
1969). 

Evaluating  games  (and  simulations)  from  the  viewpoint  of  an  analytical  scientist  is  distinct  from  assessing 
games  (artifacts)  from  the  position  of  a design  scientist.  Design  scientists  (game  designers)  build  their 
artifacts  to  function  in  well-defined  contexts  of  use  for  intended  audiences.  The  artifact  should  be  assessed 
from  that  viewpoint,  in  principle  taking  on  board  the  option  of  “causality  from  single  cases.”  Key  questions 
in  the  design  sciences  are.  Does  it  work?  Is  it  usable  in  this  context  for  this  audience?  (Klabbers,  2006,  pp. 
167-168) 

The  design  process  for  SimTIE-Math  utilized  rapid  prototyping  (Tripp  & Bichelmeyer,  1990)  to  create  and 
evaluate  prototypes  in  an  iterative  and  systematic  manner.  This  study  employs  an  examination  of  artifacts  that 
resulted  from  that  design  process,  including  design  documents,  iterations  of  game  components  and  rules,  e-mail 
exchanged  by  the  designers,  notes  from  meetings,  reports  from  playtesting  sessions,  and  other  related  documents. 
During  the  design  process,  these  artifacts  were  posted  to  a learning  management  system  (LMS)  that  was  accessible 
to  all  designers.  All  e-mail  was  exchanged  through  this  system  as  well.  Furthermore,  the  first  author,  who  served  as 
lead  designer  during  the  development  of  the  paper  prototypes,  saved  intermediate  iterations  of  game  components 
and  rules  that  had  not  been  posted  to  the  LMS.  The  authors,  who  have  continued  to  refine  the  game  while  seeking 
funding  for  development  of  a digital  version,  retrospectively  analyzed  these  artifacts  to  create  a chronology  of  the 
origins  and  evolution  of  the  game  to  its  present  state.  As  designers,  we  tried  to  determine  if  the  game  met  the  stated 
goals;  as  researchers,  we  are  interested  in  how  and  why  the  game  did  or  did  not  meet  those  goals.  By  exposing  and 
examining  the  design  process,  we  intend  to  contribute  to  precedent  in  instructional  systems  design  (Boling  & Smith, 
2008). 


Description  of  the  Project 

A review  of  the  literature  related  to  the  use  of  games  and  simulations  for  learning  indicated  that  while  there 
is  an  increasing  amount  of  research  in  this  area,  much  of  it  has  been  anecdotal  rather  than  empirical  (Dempsey, 
Rasmussen,  & Lucassen,  1996;  Hays,  2006;  Randel,  Morris,  Wetzel,  & Whitehill,  1992;  Van  Sickle,  1986). 
Furthermore,  little  research  has  been  done  regarding  the  application  of  instructional  design  theories  to  the 
development  of  games  and  simulations  for  learning  (Aldrich,  2003;  Van  Eck,  2007).  In  this  project,  instructional 
design  theories  are  explicitly  used  to  increase  the  probability  that  the  game  will  promote  the  desired  learning 
outcomes. 

The  decision  to  design  a game  as  opposed  to  some  other  medium  of  instruction  was  driven  in  part  by  the 
desire  to  increase  motivation  (Garris,  Ahlers,  & Driskell,  2002;  Malone,  1981),  engagement  (Dickey,  2005; 
Egenfeldt-Nielsen,  2005)  and  academic  learning  time  (ALT;  cf.  Berliner,  1990;  Brown  & Saks,  1986;  Kuh,  Kinzie, 
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Buckley,  Bridges,  & Hayek,  2007 ; Rangel  & Berliner,  2007)  by  providing  an  authentic  learning  experience 
(Cannon-Bowers  & Bowers,  2008;  Galarneau,  2005;  Magnussen,  2005;  Ruben,  1999).  We  know  from  research  on 
ALT  that  successful  student  engagement  in  tasks  that  are  similar  to  those  they  are  later  expected  to  perform  is 
positively  correlated  with  objective  tests  of  such  performance.  In  addition,  the  design  of  the  game  system  and  game 
components  was  informed  by  the  elaboration  theory  (Reigeluth,  1999),  first  principles  of  instruction  (Merrill,  2002; 
Merrill,  Barclay,  & van  Schaak,  2008),  theories  of  intelligence  and  learning  styles,  cognitive  load  theory  (Sweller, 
1988;  2008;  Sweller,  van  Merrienboer,  & Paas,  1998),  and  research  on  differentiated  instruction  (Hall,  Strangman, 

& Meyer,  2003;  Tomlinson,  1999). 

Initial  Conception  of  the  Simulation  Game 

The  second  author’s  idea  for  the  project  was  to  create  a simulation  game  that  would  enable  pre-service 
teachers  to  practice  integrating  technology  into  their  teaching  and  to  experience  the  consequences  of  their  decisions. 
The  project  developed  such  that  each  player  would  manage  a simulated  classroom  in  which  she  or  he  must  facilitate 
individual  student  engagement  and  learning  achievement  by  identifying  activities  and  resources  most  appropriate  for 
each  student.  The  player  would  succeed  by  most  efficiently  guiding  the  simulated  students’  mastery  of  curriculum 
standards  during  a fixed  period  of  time.  The  underlying  game  logic  would  reward  the  selection  of  activities  that 
complemented  students’  learning  needs,  styles,  and  preferences  while  utilizing  appropriate  technologies. 

The  broad  goals  of  the  project  are  to  provide  a means  for  radically  improving  pre-service  teachers’  ability 
to  individualize  instruction  through  the  use  of  appropriate  technology  and  to  advance  research  on  effective  and 
efficient  development  of  instructional  games  and  simulations.  While  the  players’  objectives  changed  as  the  designers 
refined  the  game,  the  initial  learning  goals  as  defined  by  the  second  author  persisted: 

1 . Given  an  existing  education  system,  players  will  make  changes  in  that  system  over  time  that  lead  to 
effective  technology  integration  in  that  particular  system. 

2.  Through  repeated  engagement  with  the  game,  players  will  begin  to  understand  systems  concepts  and 
apply  systems  thinking  to  the  problem  of  technology  integration  into  education. 

Description  of  Design  Process 

The  designers  initially  met  with  the  second  author  for  several  hours  over  two  days  to  clarify  needs  and 
define  deliverables  and  a timeline.  They  chose  to  follow  an  iterative  process  that  is  commonly  used  in  game  design 
as  well  as  in  design-based  research  which  consists  of  cycles  alternating  between  rapid  prototyping,  playtesting, 
evaluation,  and  revision  (Brathwaite  & Schreiber,  2009;  Design-Based  Research  Collective,  2003;  Reeves,  2000; 
Salen  & Zimmerman,  2004).  An  iterative  process  is  useful  not  only  for  improving  the  emerging  intervention  but  also 
for  identifying  reusable  design  principles  (Reeves,  2006)  and  creating  models  to  inform  design  practice  (Ma, 
Williams,  Prejean,  & Richard,  2007).  The  designers  decided  to  produce  at  least  two  paper  prototypes,  playtest  each 
prototype,  and  use  the  formative  evaluation  results  to  modify  the  subsequent  prototype.  The  final  playtest  would  also 
include  an  assessment  of  the  players'  learning. 

Given  the  paucity  of  empirically-tested  design  theories  for  instructional  games,  the  designers  decided  to 
take  an  eclectic  approach  in  which  theories  of  learning  and  instructional  design  would  be  selected  initially  to  guide 
design  decisions  and  subsequently  whenever  they  seemed  applicable  to  ensure  that  the  game  promoted  learning. 

An  initial  brainstorming  session  resulted  in  a general  structure  for  the  game  system  and  a list  of  design 
constraints  to  help  focus  the  design  problem.  The  designers  originally  envisioned  a hub-and-spoke  model  with  the 
main  game  as  the  hub  with  optional  side  games  designed  to  teach  and  reinforce  relevant  concepts  and  processes  (i.e., 
supportive  instruction  to  help  the  players  do  better  in  the  main  game).  However,  they  soon  switched  to  a model 
based  on  the  elaboration  theory  (Reigeluth,  1999).  Using  this  approach,  the  game  would  begin  with  a level  that 
offered  the  simplest  version  of  the  whole  task  (the  epitome);  subsequent  levels  would  become  increasingly  more 
complex — an  approach  common  to  videogames — with  opportunities  for  review  and  synthesis.  The  list  of  constraints 
identified  by  the  designers  included: 

• Limit  the  beginning  cast  to  one  teacher  and  3-5  students; 

• Identify  core  areas  in  which  students  must  achieve  proficiency; 

• Identify  elective  areas  in  which  different  students  must  reach  varying  levels  of  achievement,  depending  on 
the  particular  student’s  attributes  and  goals; 

• Determine  relevant  student  attributes  (which  may  include  intelligences,  learning  preferences,  attitudes, 
behaviors,  personal  goals,  etc.); 

• Provide  many  student  models  so  that  there  is  some  variety  in  the  challenges  the  player  must  meet; 
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• Identify  technologies  (hard  and  soft  combined)  and  resources  available  to  the  player; 

• Require  that  the  player  use  appropriate  technologies  to  affect  student  engagement  (academic  learning  time) 
and  student  achievement; 

• Ensure  that  the  player  has  a generous  amount  of  resources  in  order  to  simplify  learning  and  encourage 
success  in  this  initial  epitome  game. 

A couple  of  key  ideas  emerged  during  the  initial  brainstorming  session  that  persisted  throughout  the 
development  of  the  game.  The  first  suggestion  was  to  limit  the  context  of  the  game  to  a particular  grade  level.  Even 
though  this  approach  might  reduce  the  relevance  as  perceived  by  the  target  audience,  the  designers  envisioned  the 
computer  version  of  the  game  as  a shell  capable  of  accommodating  content  for  different  grade  levels.  The  second 
suggestion — which  foreshadowed  a vexing  design  problem  that  persists  to  this  day — was  to  incorporate  lesson  plans 
in  the  game.  Initially  this  was  considered  too  complicated,  but  the  designers  later  reformulated  the  idea  in  more 
abstract  terms,  not  as  specific,  detailed  lesson  plans  but  as  a more  general  game  mechanic  for  framing  the  use  of 
instructional  objectives,  activities,  and  resources.  With  the  goals  and  constraints  defined,  the  designers  decided  to 
work  individually  on  concepts  for  the  game  and  then  meet  several  days  later  to  compare  ideas  and  achieve 
consensus  on  next  steps. 


Initial  Iterations 

Two  designers  returned  with  descriptions  of  possible  games,  and  the  other  designers  contributed  ideas  for 
game  rules,  components,  and  mechanics.  Topics  of  discussion  included  whether  the  players  would  act  individually 
or  in  teams  and  whether  they  would  compete  or  cooperate  with  each  other.  One  designer  suggested  that  players  take 
turns  acting  as  teacher  and  students.  However,  others  felt  it  would  be  better  for  all  players  to  stay  in  the  role  of 
teacher.  While  discussions  about  the  structure  and  mechanics  of  the  game  continued,  there  was  general  agreement 
on  the  need  for  some  sort  of  student  model,  a curriculum  model,  an  instructional  activity  model,  a method  for 
evaluating  the  success  of  an  activity  with  one  or  more  students,  and  a method  for  tracking  student  achievement. 

The  original  conception  of  the  student  model  was  to  incorporate  Gardner’s  multiple  intelligences  (1999) 
with  student  attributes  like  socio-economic  background,  strengths,  interests,  preferences,  anxieties,  experiences,  and 
goals.  However,  the  designers  also  explored  Myers-Briggs  type  indicators  (Pearman,  Lombardo,  & Eichinger, 
2005),  Keirsey  temperaments  (Keirsey  & Bates,  1984),  Hermann  brain  dominance  instruments  (Hermann,  1990), 
and  the  Felder-Silverman  learning  styles  model  (Felder  & Silverman,  1988)  before  finally  settling  on  Kolb's 
learning  style  inventory  (Kolb,  1984),  primarily  for  the  practical  reason  that  it  is  simple  enough  to  model  in  a board 
game. 

An  early  version  of  the  student  information  card  that  the  players  would  receive  is  shown  in  Figure  1 on  the 
left  along  with  the  version  that  was  used  during  expert  review  and  playtesting  on  the  right.  These  cards  reflect  the 
decision  to  narrow  the  focus  from  general  skills  to  specific  mathematics  standards,  which  were  color-coded  on  the 
assessment  log  (described  below).  Information  on  interests/hobbies  and  friends  was  not  used  in  the  paper  prototype 
but  was  included  because  the  designers  thought  it  might  be  used  in  the  digital  version  of  the  game. 
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Aurelio  Rodriguez 

/•n 

Learning  Style 

Concrete  Reflective  +1 

AA 

Interests/ Hobbies 

Enjoys  building  models,  watching 
movies,  keeping  a weather  journal, 
playing  games. 

Friends 

Billy  Joe  Bonham 
Juan  Cortez 
Alicia  Ramos 

Skills 

C3  Math/Science  Reasoning:  1 
El  Problem  Solving:  1 
E4  Tactile/Kinesthetic:  2 

Mandatory  Elective 

E2  Interpersonal  Skills 


Aurelio  Rodriguez 

/•v 

Learning  Style 

Concrete  Reflective  +1 

AA 

Skills 


Interests/ Hobbies 

Enjoys  building  models,  watching 
movies,  keeping  a weather  journal, 
playing  games. 

Friends 

Billy  Joe  Bonham 
Juan  Cortez 
Alicia  Ramos 


Figure  1.  Two  versions  of  the  student  information  card. 


The  first  author  mapped  Keirsey’s  temperaments  and  Myers-Briggs’  types  to  Kolb’s  learning  styles  and 
then  compiled  lists  of  the  kinds  of  activities  that  would  appeal  to  each  learning  style.  This  map  guided  designers  in 
evaluating  the  effectiveness  of  an  activity  for  each  learning  style  and  assigning  bonus  points. 

The  initial  curriculum  model  consisted  of  core  competencies  (reading  comprehension,  writing  skills, 
math/science  reasoning)  and  elective  competencies  (problem  solving,  interpersonal  skills,  technology  skills, 
tactile/kinesthetic  skills,  naturalist  skills,  leadership  skills)  that  are  applicable  across  grade  levels  and  subject  areas. 
Players  would  be  required  to  foster  learning  in  the  core  competencies  for  all  students,  while  the  elective 
competencies  would  vary  by  student.  After  some  debate,  the  designers  decided  to  focus  the  prototype  on 
mathematics  at  the  fourth-  to  sixth-grade  level.  One  reason  for  this  decision  was  that  there  are  well-defined  state 
standards,  adherence  to  which  would  be  a criterion  for  evaluating  the  appropriateness  of  an  activity.  This  would 
reinforce  the  real-world  expectation  that  teachers  help  students  to  meet  those  standards.  Another  reason  was  the 
practical  matter  of  eventually  seeking  funding  to  develop  a computer  version  of  the  game.  The  designers  thought 
that  focusing  on  a high-priority  subject  area  like  mathematics  would  increase  the  chances  of  obtaining  a grant. 

Because  the  primary  game  mechanic  was  the  selection  of  appropriate  activities  for  students,  the 
instructional  activity  model  became  a crucial  element  of  the  game’s  design.  There  was  agreement  on  the  need  to 
provide  a variety  of  activities,  including  poorly  designed  activities  that  would  result  in  little  or  no  learning  for  the 
simulated  students.  However,  this  meant  that  the  players  would  have  a large  number  of  activities  to  search  through. 
The  designers  struggled  to  define  the  salient  attributes  of  an  activity  and  express  them  succinctly  to  reduce  cognitive 
load  while  maintaining  fidelity  to  the  real-world  task  being  modeled. 

The  concept  of  fidelity  in  games/simulations  seems  to  be  a point  of  contention  among  instructional 
designers,  and  it  was  at  the  center  of  many  discussions  regarding  SimTIE-Math.  Fidelity  is  the  degree  to  which  a 
simulation  is  faithful  to  that  which  it  simulates.  Reigeluth  and  Schwartz  (1989)  theorized  that  the  most  fundamental 
aspects  of  a simulation  should  have  high  fidelity,  while  lower  fidelity  is  appropriate  for  the  more  superficial  aspects 
that  may  otherwise  lead  to  cognitive  overload  and  impede  learning  and  transfer.  They  suggested  that  factors  to 
consider  include  the  complexity  of  the  real  world  environment,  the  potential  for  transfer,  the  motivational 
consequence  of  high  fidelity,  and  the  expense  of  achieving  high  fidelity.  More  recently,  Feinstein  and  Cannon 
(2002)  examined  numerous  studies  from  the  1960s  and  1970s  that  focused  on  the  effects  of  fidelity  on  training  and 
education.  They  report  that  greater  fidelity  did  not  result  in  greater  learning  and  may  in  fact  reduce  effectiveness 
through  unnecessary  complexity  and  overstimulation.  Similarly,  Winn  (2002)  notes  that  a virtual  environment  does 
not  need  to  simulate  the  real  world  to  be  useful  for  instruction,  and  that  high  fidelity  may  lead  to  constrained  and 
inflexible  understanding  and  make  it  difficult  to  transfer  knowledge  and  skills  to  new  contexts.  Ultimately  the 
designers  working  on  SimTIE-Math  decided  that  the  learning  objectives  of  the  game  determine  the  primary  game 
mechanics,  so  those  primary  game  mechanics  should  have  the  highest  fidelity  to  real-world  situations. 
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The  earliest  version  of  the  learning  activity  card  was  an  8 V2  x II  page  that  included  information  on  the 
activity’s  type  (e.g.,  group  discussion,  project),  a brief  description  of  the  activity,  the  general  skills  addressed,  the 
resources  required  (e.g.,  a computer  for  each  pair  of  students,  a calculator  for  each  student,  art  supplies;  the  amount 
of  teacher  involvement  was  also  specified),  and  an  assessment  matrix  that  indicated  which  learning  styles  would 
benefit  from  the  activity.  The  designers  found  it  difficult  to  search  though  the  pages  and  decided  to  redesign  to  fit 
the  information  on  a 4 x 6 index  card.  An  example  of  the  card  used  during  expert  review  and  initial  playtesting  is 
shown  in  Figure  2.  This  version  addresses  mathematics  standards  with  colors  corresponding  to  those  on  the 
assessment  log  (described  below).  During  gameplay,  the  success  of  an  activity  was  determined  by  rolling  a special 
die  that  would  result  in  0,  1,  or  2 points  being  awarded.  Additional  points  were  awarded  as  specified  on  the  activity 
card  if  technology  was  being  used  effectively  and  if  the  activity  corresponded  to  the  student’s  learning  style. 


TIE  Learning  Activity 

low 

low 

Type:  Problem  Solving  (Individual)- 

—Shopping  Spree 

Resources: 

• Teacher  - partial  involvement 

• 1 Computer  for  every  student 

• Internet  Access 

Description: 

Teacher  tells  students  that  they  each  have  $100  to  buy  toys. 
Students'  goal  is  to  spend  as  much  money  as  possible  without  going 
over  $100.  Teacher  gives  students  URLs  for  online  toy  stores. 
Students  may  record  purchases  on  paper  or  on  a computer  file,  but 
they  must  record  their  purchases  and  present  them  to  teacher. 


Assessment  Bonus:  Tech  +1 

Abstract  Active  +1,  Abstract  Reflective  +2 


Figure  2.  Learning  activity  card. 

The  designers  created  76  activity  cards  based  on  lesson  plans  and  activities  found  by  searching  the  Web, 
most  of  which  indicated  which  mathematics  standards  were  addressed.  The  designers  used  the  learning  styles  map 
described  above  to  decide  which  learners  would  benefit  most  from  each  activity  and  assigned  bonus  points 
accordingly.  A revised  version  of  the  learning  styles  map,  which  functioned  here  as  a design  tool,  was  provided  to 
the  players  in  the  second  playtesting  session  as  a scaffold  for  selecting  appropriate  activities  for  students. 

Scorekeeping  in  the  game  was  done  on  an  assessment  log.  Each  time  a student  participated  in  an  activity 
and  was  awarded  points  in  specific  competencies,  the  player  updated  the  student’s  log  to  indicate  the  student’s 
progress  toward  mastery.  An  early  version  of  the  assessment  log,  shown  at  the  top  of  Figure  3,  was  based  on  the 
original  approach  of  using  core  and  elective  competencies.  If  a student  failed  to  make  adequate  progress  by  the  end 
of  the  round  (achieving  from  1 to  7 points),  he  would  become  a dropout  and  be  removed  from  the  game  with  no 
points  awarded  to  the  player.  If  a student  ended  the  round  in  the  “Don’t  Graduate”  section  (achieving  from  8 to  1 1 
points),  she  would  remain  in  the  game  for  the  next  round;  the  player  would  acquire  a new  set  of  students  in  addition 
to  any  students  who  did  not  graduate,  increasing  the  difficulty  due  to  the  greater  number  of  students.  If  a student 
achieved  from  12  to  14  points,  he  would  graduate  and  be  removed  from  the  game,  and  the  player  would  score  1 
point  in  the  “Score”  column.  If  a student  achieved  15  points,  he  would  graduate  and  be  removed  from  the  game,  and 
the  player  would  score  2 points. 
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Figure  3.  Two  versions  of  the  assessment  log. 

The  version  of  the  assessment  log  used  during  expert  review  and  the  first  playtest  is  shown  at  the  bottom  of 
Figure  3.  The  shift  from  general  competencies  to  mathematics  standards  is  evident,  with  the  addition  of  color  coding 
that  is  also  used  on  the  student  information  cards  (to  indicate  a student’s  beginning  achievement  level  in  each 
standard)  and  on  the  activity  cards  (to  indicate  which  standards  are  addressed  by  the  activity).  Three  rounds  are 
indicated  with  difficulty  increasing  through  the  addition  of  standards  in  each  round.  That  is,  in  the  first  round  (the 
epitome,  in  terms  of  the  elaboration  theory)  only  Number  Sense  is  addressed,  while  in  the  second  round 
Computation  and  Algebra  & Functions  are  added;  in  the  third  round  all  standards  must  be  met.  At  the  end  of  the 
third  round,  the  player  would  tally  points  for  each  student/standard,  with  no  points  for  Low  Mastery,  1 point  for 
Partial  Mastery,  2 points  for  Near  Mastery,  and  3 points  for  Mastery.  Therefore  if  a student  achieved  Mastery  in  all  7 
standards,  the  player  would  receive  a total  of  21  points  for  that  student. 

Student  achievement  progression  changed  to  mastery  levels  so  that  the  student  was  required  to  achieve 
mastery  (21  points)  in  each  standard.  However,  there  was  some  debate  among  the  designers  regarding  the  handling 
of  students  from  round  to  round.  One  option  was  to  have  the  player  retain  the  same  students  for  each  round,  so  that 
the  only  change  was  the  addition  of  standards.  Another  option  was  similar  to  the  first,  but  the  player  also  received 
some  additional  students  to  increase  difficulty.  A third  option  was  to  tally  the  player’s  points  for  each  student  at  the 
end  of  a round,  and  then  begin  the  next  round  with  a new  and  larger  set  of  students.  This  decision  remained 
unresolved,  in  part  because  neither  playtesting  session  went  beyond  the  first  round. 

A week  before  the  first  scheduled  playtesting  session,  a full  paper  prototype  was  produced.  One  designer 
used  poster  boards  to  create  individual  “classrooms”  for  each  player  as  well  as  a shared  “computer  lab.”  The  first 
author  printed  student  information  cards  on  3 x 5 index  cards,  activity  cards  on  4 x 6 index  cards,  multiple  resource 
cards  on  8 Vi  x 1 1 paper  that  was  then  hand-cut,  learning  styles  maps,  and  job  aids  that  described  the  sequence  of 
events  in  a turn.  Small  wooden  cubes  were  used  to  create  custom  dice,  and  play  money  was  borrowed  from  a 
Monopoly  game. 

Expert  Review 

Once  a playable  paper  prototype  was  assembled,  the  designers  wanted  some  reassurance  that  the  game 
would  serve  the  needs  of  instructors  in  teacher  education.  Two  faculty  members  who  teach  education  technology 
classes  for  pre-service  teachers  agreed  to  conduct  an  expert  review  (Richey  & Klein,  2007)  which  consisted  of  a 
structured  walkthrough  of  two  turns  with  feedback  and  discussion  of  strengths,  weaknesses,  and  design  alternatives. 
Each  reviewer  was  paired  with  a member  of  the  design  team  and  together  they  examined  their  students’  attributes, 
selected  instructional  activities,  and  scored  the  results.  Other  members  of  the  design  team  observed  and  made  notes. 
The  reviewers,  who  were  experienced  school  teachers  before  becoming  professors,  were  impressed  by  the  decision 
making  required  of  the  players  and  the  fidelity  to  real-world  teaching  with  technology.  In  addition  to  numerous 
minor  suggestions  regarding  the  design  of  game  components,  key  suggestions  for  revisions  included: 

• Emphasize  the  assessment  of  technology  integration  in  the  selected  activities  to  diminish  the 
“pervasive  attitude  of  not  needing  technology”  to  teach. 

• In  addition  to  numeric  feedback  indicating  the  success  of  a selected  activity,  provide  narrative 
feedback  describing  the  outcome  to  increase  engagement. 
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• Include  an  assessment  matrix  for  each  activity  to  increase  variability  of  outcomes,  including  the 
potential  for  technical  problems  and  reward  for  anticipating  such  problems  and  arranging  a 
contingency  plan. 

Because  the  first  playtest  with  representatives  of  the  target  audience  was  scheduled  for  five  days  later,  the 
designers  decided  not  to  attempt  any  major  changes  in  the  interim.  The  first  author  agreed  to  create  a prototype  of  an 
activity  card  with  an  assessment  matrix  and  narrative  feedback  that  might  be  shown  to  playtesters  at  the  end  of  the 
session. 

Playtesting 

Playtesting  is  a methodology  commonly  used  throughout  the  game  design  process  to  systematically  test 
“gameplay,  systems,  balance,  and  interface  to  find  all  the  errors,  inconsistencies,  or  issues  and  report  them  to  the 
design  team”  (Brathwaite  & Schreiber,  2009,  p.  12).  A playtest  may  be  a full  or  partial  play  session  and.  depending 
on  the  development  stage  and  the  objectives  of  the  playtest,  may  include  the  designers,  their  friends  and  family, 
experienced  gamers,  or  “tissue  testers,”  people  who  have  never  seen  the  game  and  are  only  used  once  (Schell,  2008). 
Playtesting  is  similar  to  usability  testing,  although  Schell  (2008)  considers  the  former  to  encompass  the  overall 
experience  of  the  game  and  the  latter  to  focus  on  whether  the  interface  and  systems  are  easy  to  understand  and  use. 
The  playtesting  protocol  for  this  project  consisted  of  observation  of  the  players  combined  with  prompts  to  think 
aloud  (Preece  et  al.,  1994)  during  gameplay.  Playtesting  sessions  ended  with  informal  questioning  of  the  players  and 
the  solicitation  of  freeform  comments.  Further  details  regarding  each  playtesting  session  are  provided  below. 

Playtesting  session  1.  To  prepare  for  the  first  playtesting  session,  the  designers  used  several  approaches  to 
recruit  participants.  One  of  the  faculty  who  participated  in  the  expert  review  session  was  teaching  an  undergraduate 
teacher  education  class,  and  she  offered  her  students  extra  credit  for  participating  in  the  playtesting.  In  addition,  one 
designer  contacted  several  faculty  who  were  teaching  sections  of  an  undergraduate  class  on  technology  integration 
and  asked  them  to  encourage  their  students  to  participate.  Nevertheless,  only  three  playtesters  were  recruited:  two 
male  pre-service  teachers  in  social  studies  (one  freshman,  one  sophomore)  and  a female  graduate  student  in  math 
education. 

The  primary  goals  of  the  first  playtesting  session  were: 

• to  test  the  game  mechanics  for  usability,  cohesion,  and  “luck/skill  balance”  (Brathwaite  & Schreiber,  2009, 

P-112); 

• to  test  the  scoring  for  usability  and  timing; 

• to  get  a sense  from  the  playtesters  as  to  whether  they  found  the  game  engaging. 

At  this  point  the  designers  had  a sense  of  how  the  game  should  be  played  and  had  even  drafted  a set  of 
operational  rules  (Salen  & Zimmerman,  2004);  however,  rather  than  spend  time  assessing  the  effectiveness  of  the 
written  rules  (which  would  surely  change  through  iterations  of  the  game),  they  decided  it  would  be  best  to  explain 
the  game  and  basic  rules  and  then  let  the  players  begin  playing. 

The  first  playtesting  session  was  scheduled  for  one  hour  on  a Tuesday  evening  near  the  end  of  the  semester 
and  was  held  in  a classroom  in  the  School  of  Education.  One  designer  served  as  facilitator  of  the  playtesting  session 
while  three  other  designers  observed,  took  notes,  asked  questions,  and  occasionally  prompted  the  playtesters  to  think 
aloud  while  playing.  The  sophomore  and  the  graduate  student  played  as  a team  (Team  1)  against  the  freshman 
(Team  2).  The  facilitator  explained  the  goal  of  the  game  and  the  basic  game  mechanics  and  components.  The  game 
began  at  6:32  p.m.  and  ended  at  7:34  p.m. 

Each  team  took  seven  turns,  but  the  seventh  turn  was  perfunctory  as  each  team  had  two  students  at  mastery 
level  with  the  third  student  one  or  two  points  away  from  mastery  level.  Excluding  the  seventh  turn,  the  average 
duration  for  a turn  was  4:45  minutes,  with  the  first  turn  being  10  minutes.  Team  1 earned  a total  of  $580  from  the 
Resource  die  and  spent  a total  of  $290  on  four  resources.  Team  2 earned  a total  of  $500  and  spent  a total  of  $220. 
Both  teams  bought  Calculators  and  Art  Supplies.  Team  1 used  the  Computer  Lab  5 times  with  8 students.  Team  2 
used  the  Computer  Lab  2 times  with  5 students. 

Results  of  playtesting  session  1.  One  playtester  said  that  the  overview  of  the  game  scared  him  because  it 
seemed  like  there  was  a lot  involved.  However,  after  a couple  of  turns  he  realized  that  it  wasn’t  that  difficult.  He 
suggested  ( and  the  others  agreed)  that  the  game  should  start  with  a very  short  round  (maybe  three  turns  with  Number 
Sense  as  the  only  standard)  to  introduce  the  game,  then  move  on  to  a second  round  with  multiple  standards.  The 
players  on  Team  1 thought  that  having  two  people  on  a team  was  helpful,  and  the  player  on  Team  2 agreed  that 
having  a partner  would  have  been  useful,  in  particular  to  avoid  making  mistakes  when  matching  students  with 
activities.  Both  teams  felt  that  it  was  too  easy  to  get  money  and  suggested  raising  prices  for  resources  or  somehow 
making  resources  more  difficult  to  obtain.  Both  teams  noted  that  they  were  never  too  concerned  about  having  access 
to  the  Computer  Lab,  but  they  saw  how  it  would  be  more  of  a concern  with  three  or  four  teams  playing. 
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There  was  a significant  amount  of  discussion  regarding  scoring  and  scorekeeping.  One  playtester  suggested 
that  when  a student  moves  to  the  next  mastery  level,  he  should  stop  at  the  beginning  of  the  level.  For  example,  if  a 
student  is  near  the  end  of  Low  Mastery  and  receives  three  points  for  a Low  Mastery  activity,  he  shouldn’t  receive 
any  Partial  Mastery  points.  This  recommendation  is  consistent  with  the  idea  that  students  don’t  receive  points  for 
participating  in  activities  that  are  outside  their  current  mastery  level.  Another  playtester  suggested  making  the  Near 
Mastery  level  longer.  Her  team  had  enough  success  on  an  activity  near  the  end  that  a learner  went  from  Partial 
Mastery  to  full  Mastery.  Both  teams  thought  that  playing  a digital  version  of  the  game  would  be  more  enjoyable 
because  they  wouldn’t  have  to  spend  so  much  time  on  scorekeeping. 

In  general,  the  playtesters  characterized  the  prototype  as  a matching  game.  The  activity  cards  showed  the 
bonuses  for  particular  learning  styles  and  the  student  information  cards  showed  the  students’  learning  styles,  so  they 
simply  had  to  pair  them  while  paying  attention  to  the  standards  and  mastery  levels  addressed  by  the  activities  and 
the  resources  required.  In  other  words,  players  were  not  required  to  think  about  the  nature  of  the  activities.  One 
playtester  clearly  realized  this  after  a few  turns  and  developed  the  strategy  of  first  searching  for  activities  that 
matched  one  or  more  of  his  students’  learning  styles,  and  then  choosing  from  that  subset  an  activity  that  also 
specified  a technology  bonus.  When  shown  the  next  proposed  iteration  of  the  activity  card,  which  would  hide  the 
activity’s  association  with  learning  styles  and  the  technology  bonus  until  the  scoring  phase  of  the  turn,  the 
playtesters  thought  that  the  change  would  increase  difficulty  but  also  make  the  game  more  realistic  (i.e.,  enhance 
fidelity  of  the  primary  mechanic). 

Playtesting  session  2.  To  prepare  for  the  second  playtesting  session,  the  designers  revised  several  of  the 
game  components  and  developed  an  assessment  of  learning  to  be  administered  to  the  playtesters  before  and  after  the 
session.  The  activity  card  (see  example  in  Figure  2)  now  had  the  results  on  the  back,  with  a greater  variety  of  results 
that  nonetheless  were  more  likely  to  reward  the  selection  of  an  appropriate  activity  for  a given  learning  style  (see 
Figure  4). 


| TIE  Learning  Activity 

Type  Problem  Solving,  Practical  Application— Shopping  Spree 


Resources: 

• Teacher  Involvement:  Partial 

• Computer  for  every  student 

• Internet  Access 

Description: 

Teacher  tells  students  that  they  each  have  $100  to  buy  toys. 
Students'  goal  is  to  spend  as  much  money  as  possible  without  going 
overSiOO.  Teacher  gives  students  URLs  for  online  toy  stores. 
Students  may  record  purchases  on  paper  or  on  a computer  file,  but 
they  must  record  their  purchases  and  present  them  to  teacher. 


TIE  Learning  Activity  Results 


povTf 


Die 

Roll 

Feedback 

Total  Points 

Con 

Act 

Con 

Ref 

Abs 

Act 

Abs 

Ref 

1 

C omputer  problems  prevented  students 
from  fiiushinz 

0 

0 

0 

0 

1 

The  acti\ity  went  as  planned 

3 

2 

3 

3 

3 

Students  had  difficulties  and  needed 
assistance  Con  Act  andAbs  Act  get 
frustrated  and  receives  no  bonus 

1 

1 

l 

2 

4 

The  activity  went  as  planned 

3 

2 

3 

3 

5 

Students  had  difficulties  and  needed 
assistance  Con  Act  andAbs  Act  get 
frustrated  andreceives no  bonus 

1 

1 

i 

2 

6 

Some  students  were  motivated  to  go 
beyond  the  parameters  ofthe  assignment 
to  explore  alternatives 

3 

2 

3 

4 

Figure  4.  Revised  activity  card  (front  and  back). 


Furthermore,  there  was  now  narrative  feedback  for  each  result  and  a single  number  for  each  combination  of 
result  and  learning  style,  eliminating  the  calculations  required  in  the  previous  version  and  simplifying  scorekeeping. 
Players  would  be  told  not  to  look  at  the  backs  of  the  cards  until  they  had  paired  students  with  activities  and  rolled 
the  die  for  results.  Overall,  the  designers  thought  that  these  changes  would  reward  players  who  learned  to  associate 
certain  types  of  activities  with  learning  styles  and  who  learned  to  recognize  appropriate  use  of  technology  resources. 

A revised  version  of  the  learning  styles  map,  which  initially  functioned  as  a design  tool  for  evaluating  the 
appropriateness  of  an  activity  for  each  learning  style,  was  used  in  the  second  playtesting  session  as  a scaffold 
(Wood,  Bruner,  & Ross,  1976;  cf.  Vygotsky,  1978)  to  assist  players  in  identifying  potentially  successful  activities 
for  their  students’  learning  styles  (Figure  5). 
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Figure  5.  Learning  styles  map. 


The  descriptions  of  experiences  amenable  to  each  learning  style  suggested — but  did  not  specifically  duplicate  the 
text  of — particular  types  of  activities.  The  designers  thought  that  this  approach  would  replace  the  simple  matching 
that  occurred  in  the  first  prototype  with  more  deliberate  analysis  of  activities,  which  would  be  more  likely  to  transfer 
to  real-world  practice.  Some  minor  changes  were  made  to  the  assessment  log  (Figure  6)  based  on  the  results  of  the 
first  playtesting  session.  The  Near  Mastery  section  was  extended  and  the  Mastery  section  was  clearly  labeled  and 
assigned  a score  of  +3.  An  extra  round  was  added  so  that  a practice  round  of  a few  turns  with  one  standard  (Number 
Sense)  and  three  students  would  serve  as  a tutorial  at  the  beginning  of  the  playtesting  session. 
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Figure  6.  Revised  assessment  log. 


For  the  second  playtesting  session,  the  designers  hoped  to  recruit  eight  playtesters  to  form  four  teams.  One 
designer  made  flyers  which  were  posted  around  the  School  of  Education.  Flyers  were  also  distributed  to  faculty  who 
taught  technology  integration  classes  for  undergraduates  and  to  the  Dean’s  advisory  council,  a group  of  student 
leaders  in  the  School  of  Education.  In  addition  to  specifying  details  about  the  playtesting  session,  the  flyers 
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promised  pizza,  cookies,  and  beverages  for  participants.  A few  days  before  the  scheduled  playtesting  session,  two 
designers  spent  a couple  of  hours  in  the  School  of  Education's  atrium  distributing  information  about  the  session  to 
potential  participants. 

The  second  playtesting  session  was  scheduled  for  two  hours  on  a Wednesday  evening  two  weeks  after  the 
first  session  and  was  held  in  a classroom  in  the  School  of  Education.  Six  undergraduate  students  participated:  three 
females  (two  of  whom  were  sports  marketing  majors  while  the  third  was  a pre-service  teacher)  and  three  males  (all 
of  whom  were  pre-service  teachers). 

The  primary  goals  of  the  second  playtesting  session  were  to  test  the  revised  game  components  and  rules, 
and  to  see  if  playtesters  learned  about  learning  styles  and  differentiated  instruction  through  the  appropriate 
integration  of  technology.  Several  designers  collaborated  on  the  development  of  an  instrument  to  assess  the  players’ 
knowledge  of  technology  integration  gained  from  playing  the  game.  They  intended  to  use  this  playtesting  session  as 
a pilot  test  of  the  instrument,  which  would  then  be  refined  for  use  with  subsequent  prototypes.  Space  precludes  a 
detailed  description  of  the  instrument  and  analysis  of  the  results. 

The  playtesting  session  began  with  brief  introductions  of  the  participants,  who  were  encouraged  to  help 
themselves  to  the  food  and  drinks  that  were  provided.  The  facilitator  for  this  session  was  the  same  designer  who 
facilitated  the  first  session.  The  administration  of  the  pre-test  took  approximately  five  minutes.  The  playtesters 
formed  three  teams  with  the  help  of  the  facilitator,  who  then  spent  approximately  30  minutes  guiding  the  playtesters 
through  a couple  of  practice  turns  and  answering  their  questions.  The  playtesters  then  played  the  game  for  45 
minutes,  completed  the  post-test,  and  spent  approximately  25  minutes  discussing  the  game. 

Results  of  playtesting  session  2.  In  general  the  playtesters  found  the  game  to  be  enjoyable  and 
challenging.  As  might  be  expected  (and  desired),  the  pre-service  teachers  obtained  better  results  in  the  game  than 
students  who  were  not  studying  to  be  teachers.  The  revisions  to  the  game  components  and  rules,  along  with  the 
practice  turns,  seemed  to  reduce  the  amount  of  confusion  regarding  scorekeeping.  Because  there  were  three  teams 
this  time  instead  of  two,  each  team  had  a little  more  time  between  its  turns  to  search  for  activities.  However, 
selecting  activities  was  more  difficult  because  the  results  were  on  the  backs  of  the  cards.  Instead,  the  playtesters 
used  the  learning  styles  map  to  infer  which  activities  might  be  appropriate  for  their  students,  a process  that  took 
more  time  and  effort.  The  designers  did  not  take  detailed  notes  regarding  the  duration  of  turns  or  resources  used. 

Discussion 

We  began  this  study  by  positing  two  questions  that  we  thought  could  be  answered  through  the  design  and 
development  research  approach.  The  first  was,  how  can  instructional  design  theories  be  applied  in  the  game  design 
process?  We  based  the  structure  of  our  game  on  the  elaboration  theory  as  a way  of  ensuring  that  our  players  were 
engaging  in  whole  tasks  that  increased  in  difficulty  as  they  progressed.  While  we  only  partially  tested  gameplay  to 
validate  this  approach,  it  is  so  commonly  (if  unknowingly)  used  in  game  design  that  it  seems  a natural  application  of 
the  theory.  However,  given  a game  with  different  goals,  a different  theory  may  be  suitable.  A potential  line  of 
research  might  consider  how  game  genres  align  with  learning  objectives  and  with  instructional  design  theories. 

We  applied  Merrill's  first  principles  of  instruction  and  Kolb’s  learning  styles  to  evaluate  the  quality  of 
learning  activities  and  their  effectiveness  with  given  students.  We  would  conclude  from  this  that  instructional  design 
theories  and  learning  theories  are  applicable  as  models  for  game  components,  including  scoring  mechanisms. 

We  drew  upon  cognitive  load  theory  to  make  decisions  about  the  design  of  game  components  and  the 
amount  of  information  players  need  to  make  successful  decisions  in  the  game.  We  are  still  struggling  to  resolve 
issues  regarding  the  balance  between  cognitive  load  and  fidelity  in  the  design  of  activity  cards.  Instruction  that 
embodies  Merrill's  first  principles  should  describe  how  the  learner’s  prior  knowledge  will  be  activated,  how 
demonstration  and  practice  will  be  provided,  and  how  integration  of  the  new  knowledge  and  skills  will  be  facilitated. 
This  is  more  textual  information  than  games  generally  require  players  to  manage.  Furthermore,  at  least  in  our  current 
conception  of  the  game,  players  must  search  through  a large  number  of  activities  in  a short  time  to  keep  gameplay 
moving  forward.  There  may  be  a technological  solution  that  would  be  difficult  to  incorporate  in  a board  game,  such 
as  an  interface  for  specifying  desired  criteria  that  retrieves  only  satisfying  activities.  That  approach  would  simplify 
the  primary  game  mechanic  of  selecting  appropriate  activities  by  splitting  it  into  two  steps:  specifying  criteria  and 
examining  in  detail  the  applicable  subset  of  all  activities. 

Our  second  question  was,  what  design  principles  may  be  formulated  from  a retrospective  analysis  of  the 
design  and  development  process  and  artifacts?  It  is  clear  that  we  intuitively  decided  that  our  primary  game 
mechanic — the  selection  of  activities  that  complemented  students’  learning  needs,  styles,  and  preferences  and  that 
utilized  appropriate  technologies — needed  to  be  aligned  with  our  primary  learning  objective.  We  might  conclude 
that  designing  core  game  mechanics  that  are  aligned  with  learning  objectives  is  a fundamental  design  principle  for 
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instructional  games.  In  other  words,  if  someone  needs  to  be  able  to  do  something  in  the  real  world,  design  a way  for 
him  or  her  to  practice  that  action  and  receive  feedback  in  the  game.  Furthermore,  the  primary  game  mechanic  should 
have  high  fidelity  to  real-world  practice  to  facilitate  transfer.  Our  initial  attempt  at  this  was  flawed  and  resulted  in  a 
simple  matching  game;  we  provided  too  much  guiding  information  on  the  activity  cards  whereas  in  a real-world 
context  those  data  would  not  be  so  explicit.  However,  by  focusing  on  the  learning  objective,  we  were  able  to 
redesign  the  components  of  the  game — in  this  case  the  learning  activity  cards  and  the  student  information  cards — to 
require  the  player  to  practice  the  real  task. 

Based  on  our  experience  in  using  models  to  design  game  components,  we  would  suggest  that  when 
designing  a scoring  mechanism  based  on  the  relationship  between  two  models  (e.g.,  the  relationship  between 
learning  activities  and  learning  styles),  and  further  when  multiple  designers  are  collaborating  on  evaluating 
components  of  those  models,  that  the  process  be  treated  as  a matter  of  inter-rater  reliability.  In  our  case,  we  created  a 
map  to  assist  designers  in  making  this  judgment,  and  we  had  one  person  serve  as  final  arbiter  of  all  scoring 
decisions.  It  is  worth  noting  that  we  ended  up  giving  a version  of  that  map  to  players  as  a scaffold.  This  suggests  that 
there  may  be  a relationship  between  design  tools  and  learner  scaffolds  that  merits  further  investigation. 

As  we  reflected  on  our  design  experience,  we  noted  the  similarities  between  playtesting  and  participatory 
design  (Carr-Chellman  & Savoy,  2004).  Our  playtesters  gave  us  several  creative  and  concrete  suggestions  for 
improving  our  game,  and  this  highlights  the  usefulness  of  combining  rapid  prototyping  with  playtesting.  It  is 
increasingly  common  in  game  design  to  playtest  frequently  (cf.  the  designers’  commentary  included  in  the 
videogame  Portal,  2007;  Barnett,  Swift,  & Wolpaw,  2008),  while  in  instructional  design  the  formative  evaluation  of 
products  seems  less  frequent  and  in  many  cases  less  open  to  design  suggestions  from  participants. 
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Abstract 

An  increasing  number  of  kindergarten  through  12th  grade  students  are  using  interactive  technologies  as 
part  of  their  daily  lives;  but  its  usage  comes  at  a price.  One  of  the  costs  of  technological  interconnectedness  among 
students  is  cyber  bullying  - the  willful  and  repeated  harm  inflicted  through  the  medium  of  electronic  text  (Hinduja 
& Patchin.  2007).  Previously  Myers  and  Carper  looked  at  the  impact  of  cyber  bullying  on  schools  and  what  legal 
guidance  is  currently  available  (Myers  & Carper,  2008).  They  reported  that  cyber  attacks  affect  over  half  of  our 
teenagers  (National  Crime  Prevention  Council,  2007)  and  42  percent  of  children  in  fourth  through  eighth  grade 
(iSafe  America,  n.d.).  It  is  not  surprising  that  the  effects  of  these  attacks  transmit  over  to  performance  and  diminish 
school  safety. 

It  is  vital  that  school  personnel  clearly  understand  their  role  in  protecting  students  from  harm  while 
promoting  free  thought.  Enforcing  cyber  bullying  policies  cannot  come  at  the  expense  of  Constitutional  free  speech 
rights.  The  challenge  is  in  knowing  the  difference  and  applying  the  appropriate  response  within  the  schoolyard 
gates.  To  that  end.  this  research  involves  a survey  of  K-12  school  teachers  and  administrators  in  Illinois  and  their 
perceptions  of  abuse  of  cyber  technology  in  the  schools,  the  type  of  cyber  bullying  occurring,  and  the  actions  taken 
in  response  to  inappropriate  cyber  incidents.  The  findings  and  conclusions  of  this  study  support  both  a need  and  a 
desire  for  educational  training  on  cyber  awareness  and  cyber  policy  development.  The  results  suggest  the  need  for 
appropriate  responses  to  specific  instances  of  cyber  activity  involving  students,  school  property,  school  events,  or 
school  personnel.  Basic  guidelines  for  training  will  be  addressed  as  well  as,  the  need  for  further  research  on  the 
topic. 


Problem  and  background 

Today’s  youth  have  a great  deal  of  technological  sophistication.  They  have  embraced  cyber  technology  and 
digital  media  as  a lifestyle  for  communicating,  for  making  cultural  connections,  for  interacting,  and  for  social 
networking.  Cyber  technology  usage  allows  children  to  easily  communicate  with  their  friends  locally  and  globally. 
World  events,  scientific  breakthroughs  and  information  on  every  conceivable  subject  are  available  by  surfing  the 
Internet  as  are  services  providing  guidance,  finding  people,  locating  products,  and  purchasing  supplies  and  tickets. 
Use  of  this  technology  is  essential  for  student  success  in  today's  society.  Internet  usage  is  often  a requirement  and  a 
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necessary  tool  for  researching  and  gathering  resources  for  completion  of  school  assignments.  Although  the 
technology  has  brought  advantages  in  communication,  information,  and  entertainment  from  the  world  to  student 
fingertips,  it  has  also  brought  with  it  certain  disadvantages. 

Disadvantages  associated  with  technology  typically  include:  identity  theft,  online  predators,  pornographic 
displays,  viral  attacks,  and  spamming  (Mueller,  n.d.);  however,  school  personnel  should  be  concerned  with  the  use 
of  technology  by  children  to  harass,  humiliate,  or  bully  others  online.  It  is  this  inappropriate  usage  of  technology  that 
is  more  problematic  to  more  eight  to  15  year-  olds  than  issues  concerning  violence,  drug  and  alcohol  usage,  racism 
or  the  pressures  to  have  sex.  In  a national  survey  commissioned  by  the  Kaiser  Family  Foundation  and  Children 
Now,  55%  of  8 to  1 1 year-olds  and  68%  of  12  to  15  year-olds  reported  bullying  and  teasing  as  the  “biggest 
problem”  for  people  their  age  (“Talking  with  Kids  About  Tough  Issues”,  2001).  Among  8 to  1 1 year-olds,  only  46% 
of  them  reported  violence  as  a “big  problem”;  44%  reported  alcohol  or  drugs  as  a “big  problem”;  and  33%  reported 
pressure  to  have  sex  as  a “big  problem.”  Similarly,  in  the  12  to  15  year-  old  category,  68%  reported  alcohol  or  drugs 
to  be  a “big  problem”;  62%  reported  violence  as  a “big  problem”;  and  49%  reported  pressure  to  have  sex  as  a “big 
problem”  for  people  their  age. 

So  how  does  this  “big  problem”  relate  or  connect  to  schools,  school  administrators  and  faculty?  The  media 
is  replete  with  stories  of  cyber  aggression.  Its  extent  is  so  pervasive  that  the  Centers  for  Disease  Control  and 
Prevention  recognize  electronic  aggression  as  an  emergent  health  risk  to  students  (CDC,  n.d..  a)  affecting  nearly 
75%  of  teenagers  between  12  and  17  (Wolpert,  2008).  Further,  Wolpert  reports  on  a California  study  where  of  the 
75%  of  teenagers  who  were  bullied  online,  85%  had  been  bullied  at  school.  Other  studies  find  a minimum  of  9 to 
35%  of  all  school-aged  children  (CDC,  n.d.,  b),  and  42  to  53%  of  children  in  fourth  through  eighth  grades  have  been 
cyber  victims  (iSafe,  2006).  Every  day  160,000  children  miss  school  because  they  fear  bullying  (Shore,  n.d.). 
Associated  effects  include  emotional  distress,  depression,  suicide,  loss  of  interest  in  learning  and  conduct  problems 
including  increased  risk  of  substance  abuse,  truancy,  and  in-person  victimization.  A recent  U.S.  Secret  Service  and 
Department  of  Education  study  reported  that  71%  of  the  41  school  shooters  between  1974  and  2000  had  been  targets 
of  bullies  (Vossekuil,  Fein,  Reddy,  Borum,  & Modezeleski,  2002).  Cyber  aggression  and  its  effects  have  crept  into 
classrooms,  affecting  student  learning  and  school  administration.  Its  damaging  impact  interferes  with  student 
learning  and  prevents  students  from  achieving  their  highest  potential.  The  children  who  are  bullied,  teased,  harassed 
or  ostracized  are  not  the  only  victims.  The  bystanders  (Coloroso,  2004),  teachers  and  administrators  are  also  affected 
in  terms  of  education,  mental  and  physical  health,  and  lost  productivity,  by  the  climate  of  disrespect  and  fear  created 
by  cyber  aggression.  The  extent  to  which  schools  are  aware  of  the  problem  and  have  instituted  programs,  policies 
and  plans  to  address  this  phenomenon  is  the  focus  of  this  research. 

Methodology 

From  early  December,  2008  until  mid  January,  2009,  the  College  of  Education  and  Human  Services  at  a 
U.S.  midwestern  university  conducted  a collaborative  study  entitled  “Cyber  Technology  in  Schools:  A 21st  Century 
Dilemma.”  This  study  focused  on  cyber  bullying  and  cyber  technology  as  it  relates  to  school  policies  and  school 
training  of  K-12  administrators  and  teachers.  Four  faculty  members  (two  law  enforcement  and  justice 
administration,  one  instructional  design  and  technology  and  one  curriculum  and  instruction)  conducted  this  survey. 
Electronic  surveys  for  administrators  and  teachers  were  sent  to  the  area  districts  covered  by  nine  regional  offices  of 
education  in  the  state  of  Illinois.  A total  of  552  teachers  and  141  administrators  responded  to  the  survey.  This 
research  focused  on  the  questions  in  the  survey  that  related  to  school  personnel  perceptions  of  cyber  abuse  in  the 
schools,  types  of  cyber  bullying  occurring,  actions  taken  in  response  to  cyber  incidents  and  need  for  further 
instruction  for  handling  cyber  aggression  situations  impacting  the  educational  system. 

Survey  findings 

This  survey  was  sent  to  nine  regional  offices  of  education  in  Illinois.  One  hundred  and  forty-one 
administrators  responded  and  described  the  population  area  of  their  schools  as  71%  rural,  11%  small  city  (15,000- 
29,999),  10%  medium  city  (30,000-74,999),  4%  large  city  (75,000-150,000),  3%  metropolitan  area  (above  150,000), 
and  1%  other.  Five  hundred  and  forty-four  teachers  responded  and  described  the  population  area  of  their  school  as 
60%  rural,  13%  small  city  ( 15,000-29,999),  22%  medium  city  (30,000-74,999),  2%  large  city  (75,000-150,000),  2% 
metropolitan  area  (above  150,000), and  0%  other.  This  survey  did  not  address  the  difference  in  availability  of 
Internet  or  cell  phone  access  between  rural,  urban  and  suburban  areas  and  any  possible  impact  on  cyber  activity. 
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District  policies 

School  personnel  were  asked  whether  their  school  or  district  has  a formal  policy  for  handling  and/or 
responding  to  suspected  incidents  of  cyber  bullying.  The  responses  from  administrators  and  teachers  were 
contradictory.  As  seen  in  Figures  1 and  2,  64%  of  administrators  reported  that  their  school  had  a policy  while  81% 
of  teachers  who  responded  reported  that  their  school  had  either  no  policy  or  were  unsure  as  to  whether  a policy 
existed. 
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Figure  1.  Administrators  - policy 
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Figure  2.  Teachers  - policy 

Not  only  did  administrators  and  teachers  disagree  as  to  the  existence  of  a school  policy;  they  disagreed  as 
to  the  extent  of  training  on  electronic  aggression.  In  response  to  the  question  “Have  you  received  training  on  cyber 
bullying  and  school  behavior?”  as  displayed  in  Figures  3 and  4,  48%  of  administrators  said  they  had  received 
training  while  47%  said  they  had  not;  and  5%  were  not  sure  if  they  had  received  such  training.  Of  teachers 
responding,  17%  said  they  had  received  training;  79%  said  they  had  not;  and  4%  were  not  sure  if  they  had  received 
training. 
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Figure  3.  Administrators  - training 


Reported  incidents  of  cyber  bullying 

Despite  the  discrepancy  in  definition  of  cyber  bullying,  both  administrators  and  teachers  report  its 
occurrence  as  evidenced  in  Figures  5 and  6.  Over  45%  of  the  128  administrators  reported  having  had  a student, 
faculty  member,  or  another  administrator  comment  on  or  reported  an  incident  involving  cyber  bullying.  Only  13% 
of  the  503  teachers  responding  reported  having  had  a student,  faculty  member,  or  another  administrator  comment  on 
or  report  an  incident  involving  cyber  bullying. 
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Figure  4.  Teachers  - training 


Despite  the  reported  discrepancies  in  the  existence  of  cyber  bullying  policy  and  training,  both  administrators  and 
teachers  noted  its  occurrence.  As  seen  in  Figures  5 and  6,  over  45%  of  the  administrators  and  13%  of  teachers 
reported  receiving  comments  or  reports  of  cyber  bullying  incidents  from  a student,  faculty  member,  or  another 
administrator.  These  numbers  are  disturbing  in  light  of  the  previous  responses  concerning  policy.  Without  policy 
guidelines  and  supportive  training,  too  many  open  questions  remain.  What  behavior  is  reported?  When  is  reporting 
required?  To  whom  should  reports  be  made? 
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Figure  5.  Administrators  - reported  incidents 
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Figure  6.  Teachers  - reported  incidents 


Cyber  bullying  - an  all  encompassing  definition 

The  varied  responses  given  both  by  administrators  and  teachers  concerning  cyber  bullying  policy  and 
training  (displayed  in  Figures  1 through  4),  and  in  incident  reporting  (displayed  in  Figures  5 and  6)  indicate 
confusion  -confusion  in  terms  of  policies  in  place  to  limit  cyber  bullying  practices,  confusion  concerning  what 
activity  allows  educators  to  intervene,  and  confusion  as  to  whether  off-campus  incidents  authorize  a school  to  step  in 
and  intervene.  Part  of  the  difficulty  stems  from  the  definition  of  cyber  bullying.  The  survey  asked  the  question, 
“What  is  your  definition  of  cyber  bullying?”  Administrators  responded  at  the  rate  of  81%  (N=l  15);  teachers 
(N=378)  and  the  definitions  were  so  varied  as  to  escape  classification,  except  to  state  that  some  responses  included 
activities  protected  by  the  First  Amendment  guarantee  of  free  speech,  and  some  focused  on  criminal  activity  with  a 
wide  variety  of  definitions  in  between. 

Many  administrators  and  teachers  in  this  study  broadly  defined  cyber  bullying  pursuant  to  the  Li  (2007  ) 
definition,  using  electronic  communications  to  bully  others;  or  the  Patchin  & Hinduja  (2006)  definition,  as  “willful 
and  repeated  harm  inflicted  through  the  medium  of  electronic  text”  (p.  148).  The  responses  included  bullying 
through  e-email,  instant  messages,  social  networking  websites,  in  chat  rooms,  or  through  digital  messages  or  images 
sent  to  computers,  cellular  phones,  or  handheld  communication  devices  as  defined  by  Kowalski,  Limber,  & 

Agatston,  (2008).  Other  responses  included  any  attempt  or  intimidation  that  is  “unwelcome”,  “belittles”,  “makes 
someone  uncomfortable”,  or  “creates  a hostile  environment”  through  the  use  of  technology,  the  Internet  or  an 
electronic  device.  Also  included  were  “inappropriate  name  calling”,  “teasing”,  “making  fun  of’,  “defaming”, 
“tormenting”,  “vilifying”,  “gossiping”,  and  “spreading  rumors”  about  others. 

All  of  the  definitions  of  cyber  bullying  listed  above  have  the  potential  to  cause  harm,  to  be  hurtful,  uncivil 
or  rude,  but  not  all  may  be  regulated  by  school  policies.  Students,  like  everyone  else,  have  First  Amendment  rights 
that  school  officials  cannot  encroach.  No  precise  definition  or  universally  acceptable  policy  for  handling  online 
student  postings  has  been  proclaimed  by  statute  or  legal  precedent.  The  Supreme  Court  has  yet  to  decide  a case 
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concerning  school  censorship  of  student  expression  specifically  involving  online  postings.  Lower  courts  that  have 
addressed  the  issue  are  not  all  in  accord  (Hudson,  2000).  Individual  state  statutes  and  some  state  policies  provide 
greater  protection  of  Constitutional  provisions  than  others  for  student-generated  school  speech.  California,  for 
example,  provides  students  with  the  same  guarantees  of  free  speech  in  school  as  they  have  out  of  school  with  the 
exception  that  they  use  proper  English  (Cal.  Ed.  Code  Sections  48907  and  48950,  2006).  These  codified  variations 
make  it  more  difficult  to  differentiate  between  actionable  and  inactionable  expression  over  the  Internet  and  between 
protected  and  unprotected  speech.  Furthermore,  not  all  messages  have  the  same  impact  on  all  audiences  of  all  ages, 
gender,  and  sensitivities.  This  challenge  is  further  complicated  by  the  Internet’s  omnipresence  off  and  on  campus. 
Thus,  it  is  understandable  that  school  officials  are  confused  concerning  policies  addressing  the  situation.  The 
dilemma  becomes  - what  to  do,  when  to  intercede,  what  is  protected  and  what  is  unprotected  speech. 

Before  school  policies  can  be  created,  school  officials  must  be  taught  that  certain  categories  of  undesirable 
speech  may  not  be  regulated.  Harassment,  true  threats,  aggressive  expressions  and  stalking  may  provide  causes  of 
action,  whereas  parodies,  political  commentaries,  obnoxious  activities  or  rude  and  uncivil  behavior  may  not.  For 
example,  student-  on-  student  harassment  envelops  many  forms:  ranging  from  subtle  teasing  to  name  calling 
referencing  physical  characteristics,  racial  epithets  and  sexual  preferences.  Under  federal  law,  harassment  forms  the 
basis  of  civil  action  only  when  based  on  race,  ethnicity,  gender,  religion  or  disability  and  then  only  when  those 
harassing  acts  have  a systemic  effect  on  educational  programs  and  activities  ( Davis  v.  Monroe  County  Board  of 
Education,  1999).  School  officials  first  must  be  taught  these  subtle  distinctions  before  policy  can  be  created  and 
training  conducted. 

Speech  rights  of  students  are  “not  automatically  co-extensive  with  the  rights  of  adults  in  other  settings” 

( Bethel  v.  Fraser,  1986).  Children  are  not  given  the  same  latitude  permitted  to  adults  to  use  offensive  expression. 
Elaborate,  graphic,  and  explicit  sexual  metaphors  and  lewd,  indecent,  or  offensive  speech  by  students  may  be 
regulated  in  public  schools.  School  officials  may  protect  minors  from  exposure  to  vulgar  and  offensive  language 
including  cyber  language.  The  Supreme  Court  has  held  that  while  a school  may  not  discipline  a student  for  a 
political  viewpoint;  it  may  sanction  a student  for  making  a sexually  explicit  speech  during  a school  assembly.  This 
distinction  illustrated  in  Bethel  v.  Fraser  speaks  to  the  form  and  manner  of  student  speech,  not  its  substance.  It 
addresses  the  time,  place  and  manner  of  expression,  not  its  content  or  viewpoint.  In  a K-12  setting  "lewd,"  "vulgar," 
"indecent,"  and  "plainly  offensive"  speech  offends  and  undermines  the  school's  basic  educational  mission. 

Additionally,  student  expressions  may  be  regulated  when  the  speech  would  substantially  disrupt  or  interfere 
with  the  work  of  the  school  or  the  rights  of  other  students  ( Tinker  v Des  Moines,  1969).  Specific  and  significant  fear 
of  disruption,  not  just  some  remote  apprehension  of  disturbance  is  required.  Students  have  the  right  to  express  their 
opinions,  even  those  disliked  by  other  students  or  school  officials,  but  only  as  long  as  they  do  so  without  materially 
and  substantially  interfering  with  school  discipline  or  the  rights  of  others. 

Further,  a school  may  limit  speech  that  is  reasonably  perceived  to  be  inconsistent  with  the  shared  values  of 
a civilized  social  order.  A school  may  also  regulate  school  sponsored  speech  on  the  basis  of  any  legitimate 
pedagogical  concern  including  not  only  speech  that  would  "substantially  interfere  with  its  work  or  impinge  upon  the 
rights  of  other  students,"  ( Tinker  v.  Des  Moines,  1969  p.  509),  but  also  speech  that  is,  for  example,  “ungrammatical, 
poorly  written,  inadequately  researched,  biased  or  prejudiced,  vulgar  or  profane,  or  unsuitable  for  immature 
audiences”  ( Hazelwood  v.  Kuhlmeier,  1988,  p.  271).  Additionally,  schools  any  always  sanction  true  threats 
regardless  of  their  origin,  on  or  off  campus,  as  no  First  Amendment  protection  applies. 

Notwithstanding  state  and  federal  variations,  and  the  discord  as  to  definitions  and  decisions,  most  experts 
and  several  administrators  and  teachers  participating  in  the  survey  define  and  regulate  cyber  bullying  along  the  same 
lines  as  traditional  bullying.  The  only  difference  is  the  medium  of  expression.  They  attempt  to  resolve  cyber 
aggression  situations  by  using  the  same  methodologies  imposed  for  traditional  bullying  - fitting  the  square  peg  in 
the  round  hole.  Policies  appropriate  for  traditional  bullying  may  not  necessarily  be  appropriate  for  cyber  bullying 
due  to  differences  in  physical  location,  global  reaching,  anonymity  and  repetitive,  non-escapable  electronic 
communications.  Training  and  research  is  needed  to  distinguish  traditional  bullying  from  cyber  bullying  and 
distinguish  protected  online  expression  from  unprotected  online  behavior. 

Training  content 

Early  criticism  of  technology  usage  led  to  legislation  concerning  Internet  safety  for  children  (Children’s 
Online  Privacy  Protection  Act,  1998).  Social  networking  sites  created  policy  provisions  focusing  on  preventing 
children  from  becoming  victims  of  sexual  predators,  sexual  harassment  and  pornographic  displays.  Today, 
according  to  the  Kaiser  study  the  inappropriate  usage  of  technology  is  the  biggest  problem  facing  children;  thus, 
school  personnel  should  shift  their  concern  from  Internet  safety  to  the  use  of  technology  by  children  to  harass, 
humiliate,  or  bully  others  online.  The  shift  in  focus  from  Internet  predators  to  online  harassers  should  generate 
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further  training.  Current  policy  formulation  and  training  should  now  include  netiquette  issues  as  well  as  safety 
issues. 

The  school  surveys  produced  differing  responses  between  administrators  and  teachers  with  regard  to  the 
content  of  their  training.  Administrators  received  extensive  training  on  traditional  Internet  safety  and  sexual  predator 
issues;  but  less  training  on  netiquette  issues.  Specifically,  administrators  received  information  on  Internet  safety 
( 100  %),  sexual  predators  (84%),  sexual  harassment  (82%),  netiquette/appropriate  Internet  behavior  (74%),  other 
(2%)  and  none  (2%).  See  Figure  7.  Teachers  reported  different  findings.  Teachers  reported  training  for  Internet 
safety  (90%),  sexual  predators  (41%),  sexual  harassment  (46%),  netiquette/appropriate  Internet  behavior  (71%), 
other  (1%)  and  none  (1%).  See  Figure  8. 
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Figure  7.  Administrators  - training  content 
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Future  training 

Despite  high  responses  for  past  training  opportunities  (Figures  9 and  10),  both  teachers  and  administrators 
expressed  a desire  for  more  training  opportunities,  recognizing  the  changing  nature  of  technology,  its  uses  and  the 
legal  standards  governing  the  schools  role  in  restraining  use.  As  seen  in  Figure  10,  administrators  expressed  interest 
in  training  in  protocols  for  handling/preventing  bullying/cyber  bullying  incidents  (87%);  terminology  and  language 
specific  to  technological  communications  (66%);  discussing  cyber  bullying  and  the  school's  role  in  restraining  its 
use  (74%);  discussing  training  for  appropriate  use  of  electronic  devices  by  students  (67%).  A majority  of 
responding  teachers  responded  in  similar  fashion  (Figure  1 1).  They  said  would  be  interested  in  supporting  or 
attending  training  in  protocols  for  handling/preventing  bullying/cyber  bullying  incidents  (75%);  terminology  and 
language  specific  to  technological  communications  (50%);  discussing  cyber  bullying  and  the  school's  role  in 
restraining  its  use  (63%);  and  discussing  training  for  appropriate  use  of  electronic  devices  by  students  (55%). 
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Figure  8.  Teachers  - training  content 
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Figure  9.  Administrators  - future  training 
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The  main  finding  of  this  study  is  confusion:  confusion  over  the  definition  of  cyber  bullying,  confusion  over 
existing  policies  and  when  and  to  whom  policies  apply.  Administrators  and  teachers  need  to  know  the  distinction 
between  protected  and  non  protected  cyber  expression  and  the  different  legal  standards  that  apply.  Fortunately 
teachers  and  administrators  in  this  study  recognize  the  need  for  future  training  in  all  aspects  of  cyber  expression. 

Students  have  always  and  will  always  bully  one  another,  say  offensive  things,  and  test  the  limits  of  their 
teachers’  and  administrators’  patience  while  learning  to  express  themselves  and  develop  into  adults.  The  manner  of 
student  inappropriate  behavior  has  changed  with  the  advent  of  technology;  thus,  schools  too  must  change  to  meet  the 
new  challenges.  From  the  courts  we  have  learned  that  some  forms  of  harassing  speech  or  cyber  bullying  does  not 
warrant  First  Amendment  protection  or  school  sanctions.  Yet,  doing  nothing  is  not  an  option.  Cyber  bullying  affects 
students’  reputations  and  their  futures,  disrupts  the  educational  process  and  causes  serious  health  concerns.  So  then, 
what  must  be  done?  Again,  courts  have  provided  the  answer.  The  school's  proper  response  is  to  train  the 
professionals  to  educate  the  audience  rather  than  squelch  the  speaker. 
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Abstract 

The  purpose  of  this  presentation  is  to  highlight  some  of  the  ways  that  Web  2.0  tools  and  pragmatic  source 
ware  can  be  useful  to  student  learning.  Based  on  our  experiences  with  these  tools  and  feedback  from  our  students, 
we  also  highlight  some  of  the  pitfalls  to  avoid.  This  presentation  will  be  of  interest  to  teachers,  teacher  educators, 
media  specialists,  and  administrators  whose  goal  is  to  prepare  students  for  the  future  in  an  uncertain  global 
economy. 


Introduction 

As  new  technologies  become  available,  it  is  important  to  examine  them  to  determine  how  they  might  be 
used  to  advance  the  learning  process.  Web  2.0  technologies,  which  generally  refer  to  those  Web  applications  that 
promote  sharing  and  interactivity,  are  sometimes  thought  of  as  “new”,  but  critics  point  out  that  many  of  the 
technological  features  of  Web  2.0  were  available  in  early  web  use  (Wikipedia,  2009a).  This  is  evinced  in  the  fact 
that  approximately  two  decades  ago  users  were  able  to  “participate  and  directly  interact  and  change  web  pages”  with 
the  development  of  Really  Simple  Syndication/Rich  Site  Summary  (RSS)  robust  Technologies  in  1994  (O’Connor- 
Petruso,  2010). 

Although  diversified  definitions  of  Web  2.0  exist  today,  it  is  commonly  referred  to  as  “the  changing  trends 
in  the  use  of  World  Wide  Web  technology  and  web  design  that  aim  to  enhance  creativity,  communications,  secure 
information  sharing,  collaboration  and  functionality  of  the  web.  Web  2.0  concepts  have  led  to  the  development  and 
evolution  of  web-culture  communities  and  hosted  services,  such  as  social-networking  sites,  video  sharing  sites, 
wikis,  blogs,  and  folksonomies.”  (Wikipedia,  2009a). 

Social  networking  sites  (SNS),  or  online  communities  who  share  common  penchants  and  interact  through 
emails  and/or  blogs,  have  gained  increasing  popularity  worldwide;  they  are  usually  free,  user-friendly,  and  offer 
multiple  diversified  attributes  such  as  posting  personal  profiles,  uploading  pictures,  videos,  music,  and  so  forth. 
Users  generally  interact  through  email  and/or  blogging.  Although  analysts  disagree  as  to  the  top  twenty  most 
popular  social  networking  sites,  Facebook,  Twitter,  and  MySpace  are  usually  on  the  list  (Lewin,  2009). 

One  author  does  not  advocate  nor  use  the  SNS  in  her  classes  as  she  found  too  many  students  (both  graduate 
and  undergraduate)  were  fledglings  with  these  tools,  readily  sent  personal  information  over  the  web  and  were 
completely  unaware  of  the  lack  of  security  in  SNS.  Several  recent  startling  events  such  as  “Spyware  designed  to 
steal  Facebook  users’  information. . .Cross  site  scripting  vulnerabilities  in  Facebook. . .Twitter  accounts  used  to  send 
SPAM. . .”  are  discussed  by  experts  in  network  security  (Schultz,  2009). 

Folksonomies  are  “collaborative  tagging,  social  classification,  social  indexing,  and  social  tagging” 
(Wikipedia,  2009b).  As  these  new  and  emerging  technologies  continue  to  grow  at  a supersonic  rate  and  are  utilized 
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and  shared  on  a global  platform,  it  is  important  for  educators  and  educational  technology  specialists  to  share  their 
experiences  with  these  tools  as  there  is  a shortage  of  empirical  evidence  on  their  effectiveness. 

In  addition,  how  technology  tools  are  used  in  the  classroom  can  make  a difference  in  the  way  a learner 
learns.  Although  educators  mean  well  when  they  adopt  or  adapt  technology  tools  for  classroom  use,  sometimes  the 
technology  tools  are  not  maximally  helpful  to  the  learner  for  a variety  of  reasons.  This  presentation  will  highlight 
some  of  the  ways  that  Web  2.0  tools  can  be  useful  to  student  learning  and  some  pitfalls  to  avoid.  Similarly,  effective 
open  source  ware  will  be  discussed.  Following  are  three  groups  of  tools  we  will  highlight,  aligned  with  the 
conference  sub-theme.  Exploring  New  Media  in  Learning,  Instruction  and  Performance. 

Wikis 

Wikis  first  came  into  existence  in  1994,  when  Ward  Cunningham,  an  American  programmer,  developed  the 
WikiWikiWeb  - the  term  Wiki  orginates  from  the  Hawaiian  language  meaning  fast  and  quick  (O’Connor-Petruso, 
2010).  A wiki  is  a collaborative  interactive  community,  has  a designated  web  master,  offers  multiple  attributes  such 
as  multimedia  tools,  communication  tools,  mathematical  equations  and  spreadsheets  to  name  a few,  can  have  public 
or  private  membership,  and  be  proprietary  or  open  source.  Wikis  continues  to  emerge  and  flourish  at  a rapid  rate  - 
both  in  business  and  education. 

There  are  many  open  source  wikis,  however,  some  are  more  difficult  to  master,  are  not  very  user-friendly, 
and  offer  diversified  attributes.  After  experimenting  with  several  of  the  wikis,  one  author  found  wikispaces.com  to 
be  the  most  user-friendly  and  offer  tools  that  are  relevant  to  teachers  and  teacher  candidates. 

Over  the  past  year,  three  separate  hybrid  graduate  level  classes  at  Brooklyn  College  of  the  City  of  New 
York  (comprised  of  teachers  and  teacher  candidates)  are  utilizing  http://earlyactionresearch.wikispaces.com  (2009), 
http://actionresearchprojects.wikispaces.com  (2009),  and  http://mymstwiki.wikispaces.com  (2009).  Overall,  the 
majority  of  the  students  found  the  sites  beneficial  because  they  introduced  new  vocabulary  terms,  familiarized 
students  with  new  skills  in  technology  and/or  increased  their  present  skills,  and  provided  them  with  examples  of 
their  predecessor’s  work  on  specific  assignments.  In  addition,  many  of  these  students  turn-keyed  the  skills  they 
learned  to  their  cooperating  teachers,  students  in  the  field  setting,  and/or  students  in  their  college  classroom. 

Although  confined  to  a handful  of  students  each  semester,  a persisting  problem  one  author  still  encounters 
with  the  wikis?  is  the  students’ confusion  in  believing  they  have  “Joined”  your  specific  wiki  (ie:  MyMSTWiki  and/or 
Earlyactionresearch);  when  in  reality  these  students  have  only  created  an  account  and  become  a member  of 
wikispaces.com  - thus  becoming  a member  of  the  wiki  community  and  not  the  specific  wiki.  There  are  two  separate 
places  to  join  wikispaces  (see  red  arrows  in  Figure  1). 

Figure  1 : MyMST  Wiki  screen  shot 
http://MyMSTWiki.wikispaces.com 
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Similarly,  students  can  only  join  a specific  wiki  (red  arrow  on  left)  after  they  have  both  created  and 
activated  an  account  on  wikispaces.  Accounts  are  activated  via  email  by  confirming  wikipspaces’  invitation  to  join 
their  community.  Unfortunately  a few  of  the  students  never  respond  to  wikispaces’s  email  and  are  consequently 
prevented  from  becoming  a member.  Although  numerable  demonstrations  were  given,  the  only  remedy  for  the 
aforementioned  pitfalls  is  to  walk  these  students  “step-by-step”  through  the  process. 

Another  pitfall  is  the  issue  of  students  haphazardly  placing  their  work  in  the  wrong  area  (usually  on  the  top 
of  the  page  - see  Figure  2).  Albeit  the  professor  verbally  states  and  posts  numerable  instructions  as  to  where  to  place 
their  cursor  (in  the  next  available  cell)  before  pasting  and/or  uploading  their  document,  the  issue  remains.  To 
counteract  this  problem  of  posting  work  in  the  appropriate  area,  one  author  supplements  her  posts  with  a “hands-on” 
demonstration  and/or  individualized  instruction.  For  the  majority  of  the  cases,  the  use  of  posting  instructions, 
“hands-on”  demonstrations,  and  individualized  instruction  ameliorates  theses  issues. 
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Figure  2:  Early  Action  Research  Wiki  screen  shot 
http://earlyactionresearch.wikispaces.com 
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However  major  problems  that  affect  the  entire  class  arise  when  non-techno  savvy  students  either  do  not 
read  and/or  disregard  the  author/professor’s  instructions  and  warnings  about  Page  Updates  and  the  History  Tab.  The 
wiki  post  “The  page  has  been  updated”  informs  the  student  that  their  post  was  saved  and  the  new  page  created 
reflects  their  work.  Unfortunately  some  students  log  off  before  they  receive  a post  from  wiki  stating  that  their  “Page 
was  updated”  and  thus  become  confused  when  they  log  back  in  and  see  wiki’s  query  “Revert  to  previous  version?” 
Students  erroneously  select  “no”  and  lose  their  prior  work.  To  counteract  this  problem,  in  addition  to  posting 
instructions  and  warnings  about  Page  Updates,  a “hands-on”  demonstration  (sometimes  more  than  one)  is  given  in 
class. 

However  the  quandary  of  non  techno-savvy  students  deleting  entire  pages  when  they  go  to  post  their  work 
remains  steadfast.  Although  these  students  are  less  in  number  each  ensuing  semester,  this  problem  has  not  been 
rectified.  The  professor  has  also  abandoned  her  numerable  previous  attempts  to  show  students  how  to  use  the 
History  Tab  (which  allows  the  user  to  revert  to  previous  versions)  - as  this  tool  was  too  often  misused  by  the  non 
techno-savvy  students  and  turned  the  issue  of  one  deleted  page  into  a chronological  quagmire.  When  other  students 
saw  that  their  the  page/work  was  deleted  they  went  to  the  History  Tab  and  started  reverting  to  the  version  which 
listed  their  work  and  were  ironically  deleting  each  other’ s recent  posts.  The  only  remedy  from  this  quagmire  was  for 
the  professor  to  take  full  control  of  the  History  Tab  and  instruct  students  to  email  her  when  they  inadvertently  delete 
a page  and  then  she  alone  can  immediately  rectify  the  issue.  This  also  alleviates  the  non  techno-savvy  student  with 
the  onus  of  finding  the  previous  page. 


Blogs  and  Discussion  Boards 

In  a traditional  classroom,  the  class  ends  when  the  time  is  up  on  the  clock.  There  might  be  some  assigned 
homework,  but  the  discussion  is  cut  off  when  the  closing  bell  rings.  With  blogs  (web  logs)  and  discussion  boards, 
communication  between  class  meetings  can  continue  online. 

Further,  students  who  do  not  normally  have  a voice  in  a traditional  classroom  can  share  their  thoughts.  For 
example,  the  shy  student  may  not  feel  confident  expressing  thoughts  with  the  whole  class.  An  immigrant  student 
might  not  feel  comfortable  speaking  English  in  front  of  the  whole  class.  The  student  with  a speech  impediment  may 
be  embarrassed  to  speak.  These  students  can  take  time  to  prepare  their  ideas  before  they  are  “heard”  by  the 
classroom  community.  Sometimes  thoughtful  students  are  silent  in  the  traditional  classroom  but  appreciate  the 
opportunity  to  make  their  opinions  known. 

As  a sounding  board,  blogging  serves  its  purpose.  However  instruction  is  needed  in  “netiquette”  (net 
etiquette)  using  professional  ethics  and  standards.  Similarly,  the  instructor  must  set  parameters  and  instructional 
direction  as  student  blogging  tends  to  become  unfocused. 

We  do  not  advocate  the  use  of  blogs  for  the  aforementioned  reason.  In  addition,  blogs  do  not  always  allow 
you  to  upload/attach  files.  We  advocate  for  the  use  of  Discussion  Boards  on  course  managements  systems  such  as 
Moodle  (n.d.)  and  Sakai  CLE  (Collaboration  and  Learning  Environment)(n.d.),  (open  source  ware)  and/or 
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Blackboard  (n.d.)  (proprietary  software)  as  most  universities  subscribe  to  it.  In  addition,  both  the  professors  and 
students  can  post  topics  that  arise  in  class,  especially  if  there  is  not  enough  time  for  a full  discussion. 

Another  problem  that  arises  concerns  the  amount  of  class  credit  (or  lack  thereof)  for  posting  to  a blog  or 
discussion  board.  Our  students  are  very  busy  - almost  all  hold  teaching  positions  and  are  taking  other  classes.  To  ask 
them  to  post  their  opinions  online  without  awarding  credit  does  not  seem  to  work  well.  Students  view  posting  as  an 
extra  burden  - they  don’t  look  at  it  as  a way  to  increase  their  knowledge  base.  They  are  concerned  with  their  grades, 
but  if  this  doesn’t  “count,”  they  don’t  want  to  bother.  A recommendation  is  to  make  sure  to  award  credit  for  online 
posting. 

One  author  gives  points  for  a minimum  number  of  “substantial”  postings  throughout  the  semester.  This 
means  that  there  should  be  some  substance  to  the  posting.  (For  example,  “Yes,  I agree”  would  not  be  considered 
“substantial.”  Students  must  make  a meaningful  contribution  to  the  dialogue  for  it  to  be  counted  as  a substantial 
posting.  Of  course,  short  postings  are  permitted  but  they  don’t  count  toward  credit.  Postings  should  be  submitted 
throughout  the  semester,  otherwise  some  students  will  wait  until  the  end  to  send  all  their  postings,  thereby  reducing 
the  potential  dialogue  during  the  semester. 

One  author  uses  the  Discussion  Board  for  dialogue  on  topics  that  are  related  to  the  course  as  well  as 
commentary  on  readings  that  are  not  discussed  during  class  sessions.  The  best  dialogue  has  come  from  controversial 
issues  that  are  related  to  student  experiences.  For  example,  the  controversy  at  the  beginning  of  the  fall  2009  semester 
over  President  Obama’s  speech  to  school  children  that  was  opposed  by  some  parents  in  Texas  (McKinley  & Dillon, 
2009)  generated  a lot  of  discussion.  Although  the  course  professor  thought  there  might  be  some  differences  of 
opinion,  all  the  students  were  against  the  parental  boycott.  However,  the  discussion  was  heartfelt  and  passionate,  and 
important  points  were  made  that  were  relevant  to  the  course  objectives. 

YouTube  Videos 

Creating  YouTube  videos  under  Microsoft  Word  2007  became  markedly  easier  for  the  students  as 
compared  to  its  predecessor  MS  Word  2003.  Several  students  were  able  to  use  the  Developer  tab  and  create 
insightful  videos  on  their  projects.  Two  such  examples  of  exemplary  videos  on  the  tenets  of  the  “Water  Cycle’  - a 
major  learning  objective  from  an  elementary  science  inquiry  unit  - by  Ashley  Martin  & Bradley  Appeal  (see  Figure 
3)  and  LaShaun  Ellis  and  Cory  Feldman  (See  Figure  4)  respectively,  serve  to  enhance  learning  and  motivate 
teachers  and  teacher  candidates  to  learn  how  to  use  web  2.0  source  ware  in  their  college  classes  and  integrate  it  in  an 
elementary  curriculum. 


Figure  3:  Water  Cycle  Song  by  Ashley  Martin  & Bradley  Appel 
http://www.youtube.com/watch?v=KQ8KRznrXiA 
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Figure  4:  Water  Cycle  Boogie  by  LaShaun  Ellis  & Corey  Feldman 
http://www. youtube. com/watch?v=ipAiWXvH  24 
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The  only  issue  one  author  found  was  the  lack  of  source  ware  comparable  to  MS  Word  2007.  in  addition, 
several  students  had  issues  with  both  MS  Word  2007  and  some  of  the  newer  browsers  such  as  Bing,  Firefox,  and 
Chrome.  Several  students  found  MS  Word  2007  (precluding  the  benefits  of  the  Developer  tab)  difficult  to  navigate 
and  locate  the  desired  attribute.  Students  also  found  it  easier  to  gain  a YouTube  account  and  upload  videos  using  the 
tabs  from  Window  Explorer  as  opposed  to  Mozilla  Firefox  and/or  Google  Chrome;  perhaps  this  is  more  of  a 
YouTube  management  issue. 


Conclusion 

Although  both  these  authors/professors  have  seen  growth  in  the  technological  level  of  their  students  - 
teachers  and  teacher  candidates,  the  majority  of  their  students’  skill  levels  are  not  up  to  par  with  the  international 
community.  In  light  of  globalization  and  the  critical  necessity  to  prepare  our  students  with  “Information  Age”  skills, 
and  the  current  dire  economic  condition  that  may  hinder  updating  technological  infrastructures  and  purchasing  new 
software  and  hardware,  these  authors  believe  strongly  in  sharing  effective  Web  2.0  tools  and  source  ware  with  the 
educational  community  at  large.  Hence  these  authors  strongly  recommend  that  more  professors  and  K-12 
Technology  and  Media  Specialists  use  new  and  emerging  technologies7  with  a strong  focus  on  source  ware?  in  their 
classrooms.  Students  will  then  learn  to  adroitly  use  these  tools  and  become  competitive  citizens  in  the  global 
economy.  This  is  particularly  important  as  these  tools  continue  to  propagate  at  a cataclysmic  rate  as  the  yet 
unforeseen  horizon  of  Web  3.0  approaches. 
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The  development  of  Web-based  Instruction  model  by  using 
Knowledge  Management  Strategies  on  Educational  Technology 
Subject  for  Higher  Education  Level 

Rattakorn  Kidkarn 

Abstract 

This  research  aimed  to:  1)  develop  a Web-based  Instruction  model  by  using  knowledge  management 
strategies  on  educational  technology  subject  for  higher  education  level,  2)  test  an  efficiency  of  the  model,  and  3) 
test  an  effectiveness  of  the  model.  The  research  methods  comprised  of  three  steps:  1 ) create  a Web-based 
Instruction  model,  2)  a quality  assessment  of  the  Web-based  Instruction  model,  and  3)  the  assessment  of  the 
effectiveness  of  the  Web-based  Instruction  model  with  undergraduate  students  of  education  program  at  Nakhon 
Ratchasima  Rajabhat  University.  The  subjects  were  30  of  second  year  undergraduate  students  of  education 
program  at  Nakhon  Ratchasima  Rajabhat  University,  by  simple  random  Sampling.  T-test  dependent  used  for 
data  analysis.  The  research  results  revealed  that: 

The  Web-based  Instruction  model  using  knowledge  management  strategies  called  “DASSU  model”  includes 
5 steps  as  follows:  1)  dream  draw  (Defining:  D),  2)  find  acquisition  (Acquisition:  A),  3)  share  knowledge 
created  by  me  and  you  (Sharing:  S),  4)  keep  in  storage  (Storage:  S),  5)  use  when  necessary  (Utilization:  U).  The 
Web-based  Instruction  model  by  using  knowledge  management  strategies  on  educational  technology  subject  for 
higher  education  level  comprised  of  1 1 components:  1)  goals,  2)  objectives,  3)  subject  contents,  4)  computer 
system  and  internet,  5)  supportive  factors,  6)  a learning  process,  7)  web  interaction,  8)  learners,  9)  instructors, 
10)  specialists,  1 1)  an  evaluation.  The  educational  technology  Web-based  Instruction  model  by  using  knowledge 
management  strategies  has  an  efficiency  of  87.13/86.25,  which  is  corresponding  with  85/85  provided  criteria. 
The  assessment  of  the  effectiveness  of  the  Web-based  Instruction  model  was  found  that  the  post-test  average 
scores  in  achievement  of  students  were  statistically  significantly  higher  than  pre-test  scores  at  0.01  level,  and  the 
post-test  average  scores  on  Self-directed  Learning  ability,  were  statistically  significantly  higher  than  pre-test 
scores  at  0.01  level.  Furthermore,  the  student’s  opinions  toward  learning  through  the  Web-based  Instruction 
model  was  acceptable. 


Introduction 

Advances  in  technology  are  the  cause  of  competitions  in  terms  of  economy,  society,  culture  and  even 
education.  The  world  at  present  is  going  to  become  a society  of  knowledge  and  learning,  as  many  put  it 
"Knowledge-based  Society”  which  uses  intellectual  capital  and  knowledge  to  gain  more  advantages  in  the 
competitions.  In  order  that  a country  becomes  a society  of  good  quality  knowledge,  the  most  important  factor  is 
quality  human  resource  which  reflects  the  importance  of  education  and  the  demands  of  the  changes  as  well  as 
paradigm  in  the  modern  world.  Higher  education  plays  an  important  role  in  developing  scholars  so  that  such 
country  can  compete  with  others.  According  to  Ministry  of  Education  about  the  standards  of  higher  education, 
the  graduates  at  the  higher  education  level  must  have  knowledge  and  be  skillful  in  their  science.  They  must  be 
able  to  learn,  create  and  apply  knowledge  to  develop  themselves.  They  must  be  able  to  work  and  create  work  to 
develop  their  society  so  that  it  can  compete  with  others  in  an  international  level  (Ministry  of  Education,  2003). 
Self-directed  Learning,  therefore,  is  considered  a kind  of  learning  which  should  be  cultivated  in  students  at 
higher  education  level. 

Self-directed  Learning  is  a kind  of  learning  in  which  learners  are  responsible  for  their  plans,  practice  and 
evaluation  of  their  learning  progress.  Learning  in  this  way  can  be  done  in  every  learning  situation  because 
learners  can  transfer  what  they  learnt  and  skills  they  acquired  from  one  situation  to  another  with  meaning 
(Hiemstra,  1993).  Dixon  (1992:  2)  states  that  Self-directed  Learning  is  a process  where  learners  analyze  their 
learning  needs,  set  their  goals  of  learning,  seek  knowledge,  provide  learning  resources  as  well  as  evaluate  their 
own  learning.  These  steps  are  important  for  life-long  learning.  This  is  the  important  goal  of  students  at  higher 
education  level  (Wichai  Wongyai,  1999:  18;  Amornwit  Nakorntap,  1999:  1;  Somkit  Issarawat,  2000:  169). 

Educational  technology  is  an  important  subject  for  teacher  reformation.  According  to  the  study,  it  was  found 
that  Faculty  of  Education  in  various  universities  in  Thailand  has  been  offering  such  course  on  educational 
technology  as  a requirement  for  students  who  are  going  to  become  instructors  in  the  future  so  that  they  are 
prepared  for  tools  about  educational  technology.  They  can  develop  and  apply  technology  in  their  instruction. 
Therefore,  they  have  to  learn  principles,  theories  and  skills  essential  for  educational  technology  (Wasan  Atisap, 
2004:  11-12).  Since  the  topic  is  wide,  the  students  have  to  grasp  the  overall  concept  about  educational 
technology  first  before  they  can  go  to  other  aspects.  They  have  to  practice  how  to  use  technological  tools  and 


376 


how  to  develop  the  media  in  various  forms  according  to  the  set  topics.  There  is  less  time  for  practice  since  there 
are  many  issues  about  the  theories.  As  for  practice,  some  students  can  neither  follow  up  with  the  demonstration 
nor  review  their  practice.  They  are  not  eager  enough  to  seek  knowledge  about  both  theories  and  practice  by 
themselves.  In  order  to  make  them  aware  of  the  rapid  changes  in  technology,  an  instruction  model  for  the  course 
on  educational  technology  is  to  be  sought  in  order  to  solve  the  problems  arisen  from  such  changes.  Abilities  to 
learn  should  be  put  to  learners.  In  order  to  make  it  successful,  technology  should  be  used  because  traditional 
instruction  alone  cannot  respond  to  students  effectively  any  more.  Computer  technology  and  internet  for 
instruction  are  widely  used  and  known  such  as  online  learning,  e-learning  and  Web-based  instruction. 

Web-based  Instruction  makes  the  best  use  of  world  wide  web  in  learning.  Online  learning  plays  an  important 
role  in  changing  instruction  forms  by  using  the  potential  and  the  facility  of  technology  and  telecommunication 
system  to  facilitate  the  way  the  students  learn.  Now  they  can  learn  anywhere  anytime  without  the  limitations 
about  the  places  and  the  schedule.  Instruction  is  not  teacher-based  any  more;  it  is  now  student-centered. 
Moreover,  instead  of  learning  individually,  students  learn  in  a collaborative  manner.  They  change  their  roles 
from  waiting  for  the  knowledge  to  going  out  for  the  knowledge  (Khan,  1997).  Online  learning  has  advantages 
for  both  instructors  and  learners  (Tanomporn  (Tanpipat)  Laocharatsaeng,  2002:  18-19)  like  instructors  can 
transfer  the  contents  in  various  and  more  interesting  ways  through  multimedia.  Their  classroom  period  becomes 
shorter  so  that  instructors  have  time  to  learn  new  things.  Moreover,  instructors  can  observe  the  learning 
behaviors  and  the  progress  of  each  learner  in  details  all  the  time  through  Course  Management  tool.  Learners  can 
learn  according  to  their  own  paces.  This  is  called  “Self-paced  learning”  in  which  learners  control  their  own 
lessons  because  hypermedia  technology  is  integrated  in  such  course  with  texts,  still  images,  sounds,  graphics 
and  animations.  Learners  can  access  the  information  in  any  sequence.  Furthermore,  online  learning  helps 
facilitate  the  interaction  among  instructors,  learners  and  peers  because  there  are  many  tools  for  many  kinds  of 
interaction.  There  is  no  limitation  about  campus  so  students  from  numerous  institutes  can  come  and  share  their 
knowledge.  Despite  many  advantages,  there  are  also  disadvantages  like  students  must  have  high  responsibility 
for  their  learning.  They  have  to  be  eager  to  seek  knowledge  by  themselves.  They  must  be  able  to  work  with 
others  and  help  their  peers  in  time  of  need.  This  kind  of  instruction  is  quite  new  to  many  students,  resulting  in 
obstacles  to  them  (Wichuda  Rattanapian,  2005:  22-23).  Besides,  there  are  also  limitations  about  time  in  that 
people  are  not  free  at  the  same  time.  They  cannot  learn  at  the  same  time.  As  for  place,  virtual  places  cannot 
replace  physical  places  because  face-to-face  interaction  and  socialization  are  a part  of  building  trust  and  team 
learning.  Lack  of  face-to-face  interaction  can  lead  to  nonconformity  and  bond  in  online  community.  Learners 
might  not  be  brave  enough  to  work  and  share  knowledge  with  others  (Na  Ubon  & Kimble,  2000). 

According  to  the  problems  and  limitations  of  web-based  instruction,  many  approaches  and  methods  of 
content  presentation,  including  activities  must  be  put  on  web  sites  so  that  Web-based  Instruction  becomes 
effective  for  self-directed  learning.  This  kind  of  concept  is  related  to  Knowledge-based  economy  which  every 
organization  faces  due  to  rapid  changes  and  competitions.  One  of  the  popular  strategies  widely  acknowledged  is 
Knowledge  Management  or  KM. 

Knowledge  management  is  mainly  about  using  technique  and  various  tools  to  collect  scattered  knowledge  in 
the  same  place,  making  atmosphere  where  people  are  ready  to  think,  learn  and  build  new  bodies  of  knowledge. 
The  knowledge  is  organized  in  such  a way  that  it  can  be  retrieved  with  ease.  Moreover,  knowledge  management 
allows  people  to  have  channels  for  exchanging  knowledge  in  order  that  they  apply  the  knowledge  in  their  work 
with  success  (Davenport,  1994;  Gavin,  1994;  Prawase  Wasee,  2002;  Vicharn  Panich,  2005).  There  are  two  kinds 
of  knowledge  (Nonaka;  & Takeuchi.  2004:  53):  Explicit  knowledge  is  knowledge  which  can  be  collected,  stored 
and  transferred  easily  in  the  form  of  information  technology  or  lessons  specific  to  the  demand  of  the  learners. 
Tacit  knowledge  is  knowledge  accumulated  from  experience,  proficiency  and  skills  of  an  individual.  It  is  hard  to 
be  collected  and  stored  in  a concrete  way  because  this  kind  of  knowledge  is  normally  hidden  in  such  individual. 
Therefore,  the  mechanism  of  meeting,  trust  and  knowledge  exchange  has  to  be  done.  The  main  elements  of 
knowledge  management  are  humans,  technology,  and  knowledge  process.  Humans  are  considered  the  most 
important  element  because  they  are  the  resources  of  knowledge.  Technology  is  a tool  to  search  for,  store, 
exchange  and  make  the  best  use  of  knowledge  in  an  easy  as  well  as  fast  manner.  Knowledge  process  is  a 
strategy  for  managing  knowledge  from  users  to  users,  resulting  in  system  improvements  and  innovations.  These 
three  elements  must  be  integrated  in  balance  to  make  knowledge  management  successful.  Using  such  approach 
with  Web-based  Instruction  needs  a certain  kind  of  framework  or  a frame  of  practice  which  many  scholars  have 
already  proposed.  The  users,  therefore,  need  to  understand  and  choose  the  best  methods  according  to  the 
contexts  and  situations. 

According  to  the  problems  as  cited  above  about  the  course  on  educational  technology,  the  approach  of 
knowledge  management  along  with  learning  activities  will  help  solve  such  problems  because  knowledge 
management  strategies  will  help  learners  set  their  learning  objectives,  how  to  seek  information,  how  to  share  the 
information,  how  to  store  and  retrieve  as  well  as  make  the  best  use  of  their  knowledge.  Learners  will  become 
more  responsible  for  themselves  and  they  can  seek  the  knowledge  by  themselves.  Therefore,  the  development  of 
Web-based  Instruction  for  the  course  on  educational  technology  must  comply  with  the  knowledge  management 


377 


strategies  to  solve  the  problems  and  the  limitations  found  in  traditional  Web-based  Instruction  so  that  learners 
can  learn,  make  and  apply  the  knowledge  for  themselves.  The  results  from  this  research  will  provide  an 
understanding  of  effective  and  efficient  Web-based  Instruction  which  affects  the  learning  achievements  of 
educational  technology  course  takers  and  the  learning  abilities  of  students  at  the  higher  education  level. 

Research  Methodology 

This  research  followed  the  research  methodology  as  explained  below: 

Stage  1 The  Development  of  a Web-based  Instruction  model  by  Using  Knowledge  Management 
Strategies  on  Educational  Technology  Subject  for  Higher  Education  Level 

The  researchers  did  the  literature  review  from  various  sources  like  documents,  books,  journals  and  research 
works  in  order  to  get  an  overall  picture  of  the  approach  appropriate  for  this  research.  There  were  in  total  1 1 
kinds  of  knowledge  management  strategies  from  all  the  sources  both  in  Thailand  and  from  foreign  countries. 
The  researchers  put  all  kinds  of  them  into  a table  to  analyze  and  synthesize  the  procedure  in  knowledge 
management  strategies.  Afterwards,  the  Web-based  Instruction  was  outlined  with  knowledge  management 
strategies.  The  draft  was  reviewed  by  3 experts  in  knowledge  management  and  3 experts  in  educational 
technology.  The  model  based  on  knowledge  management  strategies  was  to  be  revised  according  to  the  experts' 
suggestions. 

Stage  2 The  Study  of  Efficiency  of  Web-based  Instruction  model  by  Using  Knowledge  Management 
Strategies  on  Educational  Technology  Subject  for  Higher  Education  Level 

After  the  revision  model  was  acceptable,  the  model  was  used  in  developing  web-based  instruction.  The 
structure  of  course  description  about  educational  technology  was  analyzed  and  included  all  the  contents  about 
Web-based  Instruction  and  behavioral  objectives  in  each  chapter.  The  web-based  instruction,  after  the  complete 
development,  was  reviewed  by  3 experts  in  educational  technology  in  order  to  check  the  accuracy  of  the 
contents  and  the  design  techniques  as  well  as  content  presentation.  The  Web-based  Instruction  was  then  tested  to 
determine  the  efficiency  whose  criteria  were  set  at  85/85.  The  test  was  done  for  3 times  with  the  sampling 
group.  In  the  first  time,  it  was  tested  with  3 students  to  check  the  basic  quality  in  terms  of  the  understanding  of 
the  contents,  the  meanings  conveyed,  the  content  presentation,  and  steps  for  activities.  The  data  from  the 
observation  and  the  interview  was  used  in  order  to  revise  the  instruction.  In  the  second  time,  it  was  done  with  9 
students  to  find  out  the  tendency  of  efficiency  of  Web-based  Instruction  and  to  find  out  the  errors.  Then  the 
instruction  was  revised.  In  the  third  time,  it  was  done  with  20  students  to  find  out  the  efficiency  of  Web-based 
Instruction  by  using  exercises  and  tests  to  measure  learning  achievements.  After  the  students  had  completed 
each  theory  lesson,  they  had  to  do  a test.  Scores  from  each  lesson  test  were  used  to  find  out  the  efficiency  (Ex). 
After  the  students  had  completed  the  course,  they  had  to  do  the  final  test,  the  score  was  used  to  find  out  the 
efficiency  (E2) 

Stage  3 The  Study  of  Effectiveness  of  Web-based  Instruction  model  by  Using  Knowledge  Management 
Strategies  on  Educational  Technology  Subject  for  Higher  Education  Level 

After  the  Web-based  Instruction  was  efficient,  the  course  was  to  be  used  by  the  second  year  students 
majoring  in  education  at  Nakhon  Ratchasima  Rajabhat  University.  There  were  240  students  who  registered  for 
this  course.  The  sampling  group  of  30  students  was  chosen  by  simple  random  sampling  method  by  lottery 
system.  The  students  did  the  pre-test  and  evaluated  their  learning  abilities  before  learning  by  themselves.  After 
the  students  had  completed  the  course,  they  did  the  final  test  and  evaluated  their  learning  abilities  again.  They 
could  express  their  opinions.  The  learning  achievement  scores  before  and  after  the  course  was  compared.  So 
were  their  learning  abilities  before  and  after  the  course.  The  opinions  toward  the  Web-based  Instruction  were 
also  collected. 


Research  Results 

The  development  of  Web-based  Instruction  model  by  using  knowledge  management  strategies  on 
educational  technology  subject  for  higher  education  level  yielded  the  following  results: 

Results  from  the  Development  of  Web-based  Instruction  model  by  Using  Knowledge  Management 
Strategies  on  Educational  Technology  Subject  for  Higher  Education  Level 

The  development  of  Web-based  Instruction  model  by  using  knowledge  management  strategies  on 
educational  technology  subject  for  higher  education  level  as  evaluated  by  experts  was  that  the  average  scores  for 
relevance  and  appropriateness  was  4.58  or  the  highest  level.  The  details  were  shown  in  Figure  1. 
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Figure  1 shows  Web-based  Instruction  model  by  using  Knowledge  Management  Strategies  on 
Educational  Technology  for  Higher  Education  Level 


Elements  of  Model 

The  model  of  Web-based  Instruction  was  developed  according  to  the  systematic  approach  consisting  of  1 1 
elements  as  follow:  1)  goal;  2)  objective;  3)  contents;  4)  computer  system  and  internet  network;  5)  supporting 
factors;  6)  instruction  process;  7)  interaction;  8)  learners;  9)  instructors;  10)  experts  in  the  course  and  11) 
evaluation.  Each  element  consisted  of  the  fallowing: 

1.  Goal  was  to  be  used  as  a norm  for  Web-based  Instruction  for  students  at  higher  education  level.  The 
idea,  the  principles  of  web  design  and  knowledge  management  were  integrated  to  help  develop  student-centered 
learning  so  that  students  have  higher  learning  achievement  for  educational  technology  subject  and  they  have 
abilities  to  learn  by  themselves  at  the  higher  education  level. 

2.  Objective  was  to  improve  learning  achievement  for  educational  technology  subject  and  develop  abilities 
to  learn  on  one's  own  at  higher  education  level. 

3.  Contents  covered  the  topics  about  educational  innovations,  learning  resources,  information  technology 
for  education  and  computer  for  instruction.  This  subject  had  both  the  contents  about  theories  and  opportunities 
to  practice. 

4.  Computer  system  and  Internet  network  consisted  of  one  server  where  the  website  was  stored.  It  was 
connected  to  clients  so  that  students  can  access  the  learning  activities  all  the  time. 

5.  Supporting  factors  consisted  of 

5.1  Webpage  which  was  considered  as  virtual  classroom  where  students,  instructors  and  experts  in  the 
course  could  meet 
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5.2  Virtual  library  was  a place  where  students  could  seek  information  about  the  contents  in  the  course 
from  all  over  the  world  through  internet  network.  This  was  facilitated  by  search  engines  on  the  web. 

5.3  Supporting  service  which  aided  communication  and  exchange  of  knowledge  through  searchable  and 
retrievable  functionalities. 

6.  Instruction  process  was  the  process  based  on  the  approach  in  Web-based  Instruction  along  with 
knowledge  management  strategies.  The  process  was  designed,  analyzed  and  synthesized  by  many  local  and 
international  scholars.  This  was  known  as  “DASSU  Model”  which  included  5 steps  as  follows:  “Dream  draw, 
Find  acquisition,  Share  knowledge  created  by  me  and  you,  Keep  in  storage,  and  Use  when  necessary”. 
The  details  were  as  follows: 

6.1.  “Dream  draw”  (Defining:  D)  was  an  activity  in  which  students  set  their  learning  goal  by 
considering  learning  objectives  of  the  subject.  Then  the  students  set  BAR  (Before  Action  Review)  together  and 
then  did  self-assessment  in  terms  of  the  level  of  knowledge  they  belong  to  and  what  elements  they  knew  and 
needed  to  know. 

6.2  “Find  acquisition”  (Acquisition:  A)  was  an  activity  in  which  students  set  approach  or  method  to 
acquire  knowledge  by  learning  plan  and  then  they  had  to  follow  the  plan  they  designed. 

6.3  “Share  knowledge  created  by  me  and  you”  (Sharing:  S)  was  an  activity  in  which  students  shared 
their  knowledge  or  what  they  knew  with  one  another.  They  could  create  knowledge  together  by  contributing  to 
Wikipedia,  blogs  and  communities  united  out  of  volition.  There  were  3 kinds  of  communities:  1)  Community  of 
Interest  (Co-I),  2)  Community  of  Practice  (Co-P),  and  3)  Community  of  Expert  (Co-E).  Moreover,  there  were  3 
strategies  to  integrate  knowledge  in  this  process  as  follows:  1)  Codified  & Personalized  Strategies  (SI),  2) 
Conversion&  Spiral  Strategies  (S2),  and  3)  Technology&  KM  Techniques  Strategies  (S3). 

6.4  “Keep  in  storage”  (Storage:  S)  was  an  activity  in  which  students  did  Benchmarking  (Bl)  to  make 
sure  that  the  knowledge  they  acquired  is  accurate  and  to  provide  Best  practice  (B2)  by  refining,  synthesizing, 
organizing  the  knowledge  from  Wikipedia,  blogs  of  friends  to  summarize  and  create  their  own  body  of 
knowledge  as  well  as  write  down  systematically  on  their  blogs. 

6.5  “Use  when  necessary”  (Utilization  : U)  was  activity  in  which  students  tested  their  knowledge 
through  activities  in  each  topic.  They  would  know  their  level  of  knowledge  and  make  the  best  use  of  their 
knowledge.  This  was  called  AAR  or  “After  Action  Review”  to  summarize  and  evaluate  the  acquired  knowledge 
together  in  groups.  They  would  determine  whether  the  knowledge  meets  the  goal  and  why.  They  would  find 
solutions  for  further  improvements. 

7.  Interaction  was  an  activity  among  members  in  the  group  of  the  same  interest  in  which  they  discussed  the 
topics,  made  contacts  with  instructors  and  experts  in  the  course  through  website. 

8.  Learners  had  an  important  role  in  self-directed  learning,  especially  Web-based  Instruction  because  they 
had  to  share  their  knowledge  and  follow  the  activities  offered  in  the  course.  They  could  ask  for  information  and 
suggestion  with  instructors  and  experts  in  the  course  as  well  as  their  peers. 

9.  Instructors  had  a role  to  facilitate  the  learning  of  learners.  They  provided  the  students  with  tools  ready  to 
be  used.  They  also  tracked  down  and  observed  the  way  the  students  learn  online  so  that  they  could  come  to  help 
in  the  time  of  need  when  students  found  difficulties  with  web-based  instruction.  They  also  guided  and  motivated 
students  to  learn  and  complete  all  learning  activities. 

10.  Experts  in  the  course  were  experts  in  educational  technology.  They  were  chosen  out  of  the  lecturers 
who  were  specialized  in  educational  technology.  They  made  the  best  use  of  their  knowledge  and  experience  in 
assigning  the  tasks  suitable  for  each  individual  in  each  topic.  They  gave  guidance  to  students  in  every  step  of  the 
learning  process. 

11.  Evaluation  was  a kind  of  assessment  done  online.  The  learning  achievement  test  was  done  before  and 
after  the  course  on  educational  technology  in  terms  of  knowledge,  understanding,  and  application.  The  test 
consisted  of  40  objective  questions,  each  of  which  had  4 multiple  choices.  The  learning  achievement  scores 
before  and  after  the  course  were  compared  and  evaluated  through  self-assessment  tools  and  the  students’ 
opinion  towards  Web-based  Instruction  model.  Students'  behaviors  and  participation  were  also  observed  and 
evaluated  according  to  the  Web-based  Instruction  model. 

Results  from  the  Study  of  Efficiency  of  Web-based  Instruction  model  by  Using  Knowledge 
Management  Strategies  on  Educational  Technology  Subject  for  Higher  Education  Level 

The  development  of  Web-based  Instruction  model  by  using  knowledge  management  strategies  on 
educational  technology  subject  for  higher  education  level  as  evaluated  by  experts  yielded  the  following  results. 
The  average  score  for  the  content  suitability  was  4.36  or  at  high  level.  The  average  score  for  the  design  and 
presentation  technique  was  4.38  or  at  high  level.  The  test  to  find  out  effectiveness  yielded  the  score  of 
87.13/86.25.  To  conclude,  the  Web-based  Instruction  model  by  using  knowledge  management  strategies  on 
educational  technology  subject  for  higher  education  level  corresponded  with  85/85  provided  criteria.  That  is  to 
say,  the  model  was  efficient  and  suitable  for  students  at  higher  education  level. 

Results  from  the  Study  of  Effectiveness  of  Web-based  Instruction  model  by  Using  Knowledge 
Management  Strategies  on  Educational  Technology  Subject  for  Higher  Education  Level 
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It  was  found  that  the  pre-test  and  post-test  average  scores  in  achievement  of  students  were  statistically 
significantly  different  at  0.01  level.  The  post-test  average  scores  in  achievement  of  students  were  higher.  The 
ability  to  learn  on  one’s  own  was  also  improved.  The  students'  opinion  towards  learning  through  the  Web-based 
Instruction  model  was  4.32  on  average  or  acceptable. 

Discussions 

According  to  the  development  of  Web-based  Instruction  model  by  using  knowledge  management  strategies 
on  educational  technology  subject  for  higher  education  level,  there  were  2 issues  to  be  discussed:  1)  the 
development  of  Web-based  Instruction  and  2)  the  study  of  efficiency  of  the  learning  through  web-based 
instruction. 

1.  As  for  the  development  of  Web-based  Instruction  model  by  using  knowledge  management  strategies  on 
educational  technology  subject  for  higher  education  level,  the  issues  to  be  discussed  were  given  below: 

1.1  The  results  from  the  development  of  Web-based  Instruction  model  by  using  knowledge 
management  strategies  on  educational  technology  subject  for  higher  education  level  yielded  the  supporting 
factors  as  follows: 

1)  The  development  of  Web-based  Instruction  model  in  a systematical  manner.  The  web  site  was 
developed  systematically  and  processed  in  a step-by-step  manner  by  analysis  and  synthesis  of  the  traditional 
instruction.  The  philosophy  in  education  was  also  applied  to  emphasize  on  the  self-directed  learning,  in  which 
the  students  were  the  one  to  create  knowledge.  Other  theories  used  in  the  course  were  theories  about 
behaviorism,  motivation,  approach  in  web-based  instruction,  knowledge  management  strategies.  The  elements 
of  Web-based  Instruction  were  analyzed  and  synthesized  into  the  model  which  was  revised  many  times 
according  to  the  suggestions  by  experts.  The  model  was  tested  for  the  efficiency  before  real  use. 

2)  Instruction  process  consisted  of  5 activities  which  came  from  analysis  and  synthesis  of 
knowledge  management  strategies  proposed  by  scholars  in  Thailand  and  abroad.  The  results  from  the  synthesis 
were  used  to  provide  students  with  activities  suitable  for  web-based  instruction.  Those  were  defining, 
acquisition,  sharing,  storage,  and  utilization.  To  make  them  easy  to  remember,  the  activities  were  given  new 
names  as  follows:  “Dream  draw,  Find  acquisition,  Share  knowledge  created  by  me  and  you,  Keep  in 
storage,  and  Use  when  necessary”.  The  steps  corresponded  with  the  student-based  instruction  in  which 
students  were  the  center  of  the  instruction.  The  students  know  themselves,  know  the  world  and  depend  on 
themselves  in  terms  of  economy,  spirit,  society  and  learning.  They  were  happy  to  learn  (Prawase  Wasee,  1999). 
This  corresponds  with  the  Self-directed  Learning  in  which  the  students  set  their  goal  and  plan  the  learning  path 
by  themselves.  They  control  and  evaluate  themselves  by  seeking  the  methods  suitable  for  their  success  in 
learning  (Oladoke,  2006:  13) 

1.2  According  to  the  test  to  find  out  the  efficiency  of  the  Web-based  Instruction  model  by  using 
knowledge  management  strategies  on  educational  technology  subject  for  higher  education  level,  Ei  /E2  were 
87.13/86.25,  higher  than  85/85  provided  criteria.  This  was  because  the  model  was  designed  in  a systematic  order 
and  revised  many  times  by  experts.  Therefore,  the  efficiency  of  the  Web-based  Instruction  was  acceptable. 

2.  As  for  the  study  of  the  Web-based  Instruction  model  by  using  knowledge  management  strategies  on 
educational  technology  subject  for  higher  education  level,  it  was  found  as  follows: 

2.1  The  post-test  average  scores  in  achievement  were  higher  than  the  pre-test  average  scores  in 
achievement.  This  indicates  that  the  instruction  model  could  be  used  as  a tool  to  facilitate  learning  because  the 
Web-based  Instruction  was  developed  in  a systematic  order,  allowing  all  students  to  participate  in  studying  and 
doing  revision  for  all  the  topics  from  seeking  the  contents  to  posting  in  Wikipedia.  This  corresponds  with  the 
strategies  proposed  by  Hansen  & et  al  (2004)  in  that  codified  strategy  is  suitable  for  explicit  knowledge.  The 
organization  with  a server  benefits  from  this  by  encoding  knowledge  and  storing  it  in  an  accessible  database 
where  everybody  can  get  connected.  Due  to  the  limitation  that  the  contents  to  be  posted  had  to  be  different  from 
others,  the  students  had  to  read  all  the  information  first  before  posting.  This  way,  the  students  got  higher  post- 
test average  scores  in  achievement. 

2.2  The  learning  abilities  of  students  were  improved  after  the  course  indicated  that  the  instruction  model 
helped  the  students  to  improve  their  learning  abilities  due  to  the  following  reasons. 

2.2.1  The  instruction  model  was  based  on  knowledge  management  strategies  and  included  5 
activities.  This  was  done  through  analysis  and  synthesis  proposed  by  scholars  in  Thailand  and  abroad.  The 
activities  focused  on  the  participation  by  students  from  the  first  stage  like  setting  their  goal,  defining  how  to 
learn,  evaluating  what  they  already  know,  and  planning  to  the  last  stage  like  following  the  plan  and  evaluating. 
The  students  were  responsible  for  their  own  learning.  Their  practice  was  recorded  in  portfolio  and  posted  in 
blogs,  allowing  other  students  to  do  a revision  and  correct  their  goal  according  to  their  plans. 

2.2.2  As  for  the  application  of  technology  and  technique  in  knowledge  management,  the  technology 
used  consisted  of  Course  Management  Systems  (CMS),  webboard,  chatroom,  and  blogs.  These  tools  were 
learning  resources  and  factors  for  accessibility  as  well  as  interaction  between  instructors  and  students  and 
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among  students.  This  technology  resolved  the  limitations  of  students  in  terms  of  time  and  place.  They  could  also 
chat  online  to  discuss  at  the  same  time,  increasing  the  level  of  interaction.  They  could  trust  and  confide  in  the 
online  community.  They  were  also  voluntary  in  working  together  as  well  as  sharing  information.  Knowledge 
management  technique  which  reduced  the  instructional  limitations  consisted  of  2 things:  First,  process 
management  which  is  to  make  sharing  and  applying  knowledge  easier  to  be  done  by  the  most  of  the  participants 
through  awards  like  gifts  and  scores;  Second,  space  management  which  is  to  create  an  atmosphere  where 
students  can  know  and  trust  one  another  in  the  end.  This  idea  corresponds  with  Na  Ubon  & Kimble  (2002)  in 
that  confiding  in  each  other  is  the  base  of  sharing  knowledge  which  leads  to  effective  knowledge  sharing  and 
transferring. 

2.2.3  Knowledge  could  be  shared  by  all  participants.  The  atmosphere  facilitates  the  learning  through 
online  community.  There  were  many  activities  and  people  shared  knowledge  with  all  members  in  chatroom 
through  knowledge  conversion  process  and  knowledge  spiral  according  to  SECI  Model  proposed  by  Nonaka  & 
Takeuchi  (1995).  This  model  was  adopted  in  supporting  education  through  self-learning  educational  technology. 
The  students  could  learn  individually  and  learn  in  group  (group-learning).  This  was  to  transform  tacit 
knowledge  to  explicit  knowledge  by  using  computer  as  a tool.  Students  could  learn  by  various  methods, 
increasing  their  achievement  and  learning  abilities  after  the  course. 

Suggestions 

The  Web-based  Instruction  model  by  using  knowledge  management  strategies  on  educational  technology 
subject  for  higher  education  level  was  developed  with  the  approach  that  all  students  are  able  to  improve 
themselves.  Learners  are  the  most  important  element.  Therefore,  instruction  should  correspond  with  their 
preference  and  skills  by  considering  their  differences  among  them.  This  way,  students  will  have  abilities  to  learn 
by  themselves.  The  approach  meets  the  trend  of  current  education  which  focuses  on  life-long  learning. 

1.  Suggestions  for  Using  this  Instruction  Model 

1.1  The  application  of  Web-based  Instruction  should  be  based  on  the  readiness  of  many  elements, 
especially  the  readiness  of  students.  There  should  be  an  orientation  for  students  so  that  they  understand  the 
objectives  of  learning  in  this  way.  The  students  also  have  more  responsibilities  in  their  learning.  They  should 
know  the  benefits  and  have  to  get  familiar  with  the  technology.  The  instructors  need  to  prepare  themselves  too. 
They  must  provide  students  with  time  for  guidance  and  help.  They  have  to  know  how  to  use  the  tools  for 
instruction.  Computers  and  server  are  also  important.  They  must  be  ready  for  use  all  the  time. 

1.2  To  make  the  instruction  model  effective,  it  can  be  used  both  online  and  face-to-face,  or  mixed. 
However,  the  activities  should  be  done  in  the  organized  steps. 

1.3  To  apply  the  instruction  model  with  various  factors  like  space,  time,  event  and  people  (STEP: 
Space:  S,  Time:  T,  Event:  E,  People:  P),  the  space  or  place  to  be  used  can  be  real  or  virtual.  If  the  space  is 
virtual  and  people  are  not  free  at  the  same  time,  the  exchange  of  knowledge  cannot  be  done.  Time  to  engage  in 
the  activities  is,  therefore,  essential.  As  for  event  and  people,  the  model  should  be  adapted  to  correspond  with 
the  event  and  the  group  of  people. 

1 .4  The  roles  and  the  responsibility  of  members  should  be  clarified  so  that  they  are  ready  for  the  proper 
use  of  the  technology,  making  the  instruction  model  effective  to  the  maximum. 

2.  Suggestions  for  Further  Studies 

For  further  studies,  the  researcher  gives  the  following  suggestions. 

2.1  The  Web-based  Instruction  model  should  be  applied  in  other  subjects  or  courses  and  then  the 
comparison  of  the  permanence  of  the  model  is  done  so  that  we  achieve  the  general  instruction  model  by  using 
knowledge  management  strategies. 

2.2  The  effectiveness  of  the  instruction  model  in  terms  of  changed  learning  behaviors  should  be  studied 
by  using  knowledge  management  strategies  and  learning  styles  in  real  life. 
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Abstract 

A common  challenge  within  large  online  learning  or  training  environments  is  helping  the  user  find 
information  that  is  useful  and  relevant  to  their  needs.  Recommendation  systems  have  been  used  successfully  in  E- 
commerce  sites  (e.g.  Amazon.com)  to  help  customers  find  products  they  may  be  interested  in,  however,  their 
potential  uses  within  online  learning  environments  have  not  been  fully  explored.  An  exploratory  study  conducted  at 
a large  Midwestern  university  surveyed  students  using  a recommendation  system  to  provide  recommendations  of 
discussion  board  activity  occurring  in  their  online  course  site.  Results  indicated  that  while  students  appeared  to  find 
the  recommendation  systems  easy  to  use,  and  largely  understood  what  the  recommendations  were,  many  did  not  find 
the  recommendations  provided  all  that  useful.  The  author  proposes  that  additional  research  be  undertaken  to  fully 
understand  the  nature  of  activities  students  find  important,  and  how  to  tune  recommendation  systems  to  match  this 
paradigm  within  online  learning  environments. 


Introduction 


Background 

The  rapid  growth  of  the  use  of  online  learning  environments  to  provide  training  and  instruction  creates  new 
challenges  when  trying  to  determine  methods  to  make  online  interaction  and  collaboration  between  participants 
easier  and  more  effective.  A key  requirement  to  effective  online  interaction  is  being  aware  of  what  has,  and  what  is 
currently  happening  in  your  environment  (Carroll  et.  al,  2002).  As  Moody  (2000)  argues,  this  awareness  of  activity 
in  our  environment  is  a precursor  to  any  type  or  level  of  interaction  between  participants.  Despite  this  importance, 
many  popular  learning  environments  in  use  today  (such  as  Blackboard  or  Sakai)  are  severely  lacking  in  any  type  of 
awareness  or  notification  tools.  In  an  attempt  to  remedy  these  shortfalls,  middleware  tools  such  as  the  Context-aware 
Notification  System  (CANS)  have  been  developed.  CANS  was  designed  to  provide  a centralized  awareness 
notification  tool  for  a multitude  of  external  environments  (Amelung,  C.,  2007).  CANS  works  by  monitoring 
specific  events  (i.e.  a discussion  board  response,  or  a change  to  a document)  that  occur  within  an  environment,  and 
then  refining  these  events  into  visible  notifications  to  the  end  user. 

However,  one  issue  that  soon  arises  from  presenting  all  potential  activity  notification  information  directly  to 
the  participants  is  that  in  a busy  online  learning  site,  the  amount  of  notifications  can  grow  extremely  quickly.  As 
Jacoby  ( 1977)  indicates  in  his  article  on  information  load,  it  can  be  difficult  for  a user  to  identify  relevant 
information  after  their  information  processing  capacity  is  reached.  A natural,  and  common  solution  to  the 
information  overload  challenge  caused  by  “excessive”  notifications  is  to  simply  aggregate  the  notification 
information.  Thus,  instead  of  showing  individual  notices  of  each  event;  the  system  instead  displays  a summary  of 
how  many  events  occurred  of  a certain  type  (i.e.  There  are  21  new  discussion  posts  since  yesterday).  However,  a 
major  drawback  to  this  aggregation  model  is  that  users  lose  all  the  important  data  elements  from  the  notification.  In 
other  words,  the  user  can  no  longer  determine  the  nature  or  location  of  the  activity  happening  within  their 
environment;  rather  they  only  can  ascertain  that  some  activity  has  occurred  somewhere  in  the  site.  The  user  is  still 
left  to  their  own  devices  to  search  out  where  this  activity  is  occurring,  and  whether  or  not  it  is  of  interest  to  them. 

Thus,  another  potential  solution  that  could  help  reduce  information  overload,  while  still  keeping  the  rich 
notification  data  intact,  is  the  use  of  a recommendation  system  (Good  et.  al,  1999)  for  activity  notification.  Using 
techniques  of  knowledge  discovery,  a recommendation  system  attempts  to  generalize  user  preferences,  compare  this 
knowledge  with  what  is  known  of  the  dataset,  and  then  make  a personalized  recommendation  to  the  user  (Sarwar  et. 
al,  2001).  For  activity  notification,  a recommender  system  would  take  the  past  behavior  of  a user  (e.g.  where  they 
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visited,  what  they  read,  who  they  responded  to)  and  compare  it  to  other  user  behavior  within  the  system.  This 
comparison  would  then  yield  a more  personalized  list  of  activities  to  the  user,  based  on  prior  behavior  (Haubl  and 
Murray,  2006). 

Purpose  of  Study 

Although  recommender  systems  have  been  successfully  integrated  and  utilized  in  various  other  domains  such 
as  eCommerce  applications  (e.g.  Amazon)  and  social  sites  (e.g.  MovieLens),  there  is  still  much  work  to  be  done  in 
applying  recommendation  systems  to  e-leaming  (Zaiane,  2002).  The  purpose  of  this  study  is  to  learn  more  about  the 
student  perceptions  of  using  recommendation  systems  within  an  online  learning  environment.  For  this  exploratory 
study,  we  are  primarily  concerned  with  three  questions: 

1)  To  what  extent  will  students  find  the  recommendation  system  easy  to  use? 

2)  Do  the  students  perceive  the  recommendation  system  as  useful/helpful? 

3)  To  what  degree  do  the  students  become  “attached”  to  having  recommendations  provided  in  their  course? 

Methods 


Context 


The  research  study  took  place  at  a large  Midwestern  University,  using  online  courses  taught  on  the  Sakai 
Collaborative  Learning  Environment  (Sakai).  Each  course  consisted  of  at  least  20  students,  and  was  taught 
primarily  online.  Membership  in  these  courses  was  mostly  graduate  level  students,  however  some  undergraduate 
students  were  also  enrolled.  Instructors  were  recruited  to  be  apart  of  the  pilot  study,  which  required  them  to  modify 
their  course  home  page  to  include  a “Social  Widget”  which  contained  the  recommended  discussion  activity. 

Social  Widget 

To  provide  recommendations  for  students,  a custom  Flash-based  Social  Widget  was  constructed  using  the 
Flex  development  environment.  The  Social  Widget  included  three  sections:  a social  comparison  chart,  a list  of  hot 
topics,  and  recommended  discussions  (see  Figure  1). 


Figure  1:  A screenshot  of  the 


“Social  Widget”  tool  that  was  added  to  each  course  Home  Page. 
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Students  were  provided  recommendations  in  the  third  section  of  this  Social  Widget  entitled:  “Discussion 
that  may  interest  you”.  This  section  consisted  of  a discussion  topic  title  that  the  student  was  recommended,  as  well 
as  a “match”  number  that  indicated  how  well  this  particular  recommendation  matched  the  user.  Clicking  on  the 
topic  link  would  launch  the  recommended  discussion  topic  in  a new  browser  window.  The  match  number  and 
recommended  discussion  topics  were  based  on  a student’s  individual  historical  activity  on  the  discussion  boards. 

For  this  study,  the  Taste  collaborative  filtering  engine  (part  of  Apache’s  Mahout  project)  was  used  to 
provide  the  recommends.  Each  day,  a student’s  historical  discussion  board  activity  was  obtained  from  the  Context- 
Aware  Notification  System  (CANS)  and  input  into  the  Taste  collaborative  filtering  engine.  The  Taste  engine  would 
analyze  the  activity  data,  and  output  a list  of  recommended  discussion  posts  for  each  student.  These 
recommendations  were  then  added  into  the  Social  Widget  to  be  displayed  to  the  user  each  morning. 

Study  Procedure 

At  the  beginning  of  a semester,  the  “Social  Widget”  was  added  to  the  Home  Page  of  4 different  online 
courses  taught  on  Sakai.  The  students  were  not  instructed  on  how  to  use  the  social  widget,  it  just  merely  appeared 
to  them  as  an  additional  tool  to  use.  The  Social  Widget  did  contain  a small  Help  page,  which  the  students  could 
click  on  to  view  more  about  the  widget. 

After  approximately  1 month,  we  sent  out  a recruitment  email  to  each  of  the  96  students  enrolled  in  our 
four  test  courses,  asking  for  volunteers  to  take  a short  survey  relating  to  their  experiences  using  the  recommendation 
system,  and  the  other  social  tools  within  Sakai. 

The  survey  included  10  questions  on  the  recommendation  system,  and  attempted  to  measure  the  following 
three  constructs:  usefulness,  ease  of  use,  and  attachment.  Survey  questions  for  usefulness,  and  ease  of  use  were 
adapted  from  the  Venkatesh  and  Davis  (2000)  TAM2  Measurement  Scale,  while  questions  for  attachment  were 
adapted  from  Madsen  and  Gregor  (2000)  Human-Computer  Trust  scale.  Respondents  were  asked  to  rate  each 
question  using  a Likert  scale  where  1=  strongly  agree,  2 = agree,  3 = kind  of  agree,  4 = neutral,  5 = kind  of  disagree, 
6 = disagree,  and  7 = strongly  disagree.  Of  the  96  invitations  sent,  we  received  45  responses  to  the  survey. 

Results  & Discussion 


Ease  of  Use 

The  survey  results  came  back  favorably  when  the  students  were  asked  about  the  difficulty  of  using  the 
recommendation  portion  of  the  Social  Widget.  When  we  combine  the  survey  questions  dealing  with  ease  of  use, 
we  see  over  58%  of  the  students  at  least  somewhat  perceived  the  recommendation  system  was  easy  to  use,  34% 
were  neutral  to  the  system,  with  the  remaining  8%  indicating  it  was  not  easy  to  use  (see  Figure  2). 

Figure  2:  Frequency  distribution  of  respondents  to  the  combined  survey  questions  regarding  “Ease  of  Use” 
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This  result  was  not  surprising,  as  the  recommendation  system  was  extremely  straightforward,  and 
specifically  designed  to  be  passive  (e.g.  the  students  were  not  required  to  “rate”  discussions)  so  as  to  minimize  the 
time  and  mental  effort  required  to  use  it.  In  addition,  some  students  may  have  already  had  interaction  with 
recommendation  systems  on  popular  ecommerce  sites  like  amazon.com,  or  movie  rental  sites  like  Netflix,  and  thus 
may  already  have  a well-established  mental  model  of  the  purpose  and  use  of  a recommendation  system. 

Usefulness 


While  the  majority  of  students  found  the  recommendations  easy  to  use,  only  about  38%  indicated  that  the 
recommendations  were  at  least  somewhat  useful.  Conversely,  39%  of  the  respondents  at  least  mildly  disagreed 
with  the  survey  questions  dealing  with  usefulness,  with  the  other  23%  as  a neutral  response  (see  Figure  3). 


Figure  3:  Frequency  distribution  of  respondents  to  the  combined  survey  questions  regarding  “Usefulness 
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The  responses  to  our  usefulness  questions  indicate  that  while  some  people  found  the  recommendations 
useful,  the  vast  majority  were  either  indifferent,  or  didn’t  find  the  recommends  all  that  useful.  From  these  results, 
we  might  interpret  that  the  discussions  recommended  were  not  the  discussions  the  students  “wanted”  to  see.  One 
explanation  for  this  is  the  recommendation  system  did  not  have  enough  activity  data  on  some  of  the  users  to  provide 
good  recommends.  Another  explanation  could  be  the  recommendation  system  treated  every  discussion  “equally”, 
yet  in  an  online  learning  site,  this  is  not  typically  the  case.  Learners  may  value  posts  from  the  instructor/trainer 
more  highly,  and/or  perhaps  posts  relevant  to  specific  assignments.  In  this  manner,  the  recommendation  systems 
included  in  an  online  learning  environment  may  need  to  know  more  about  the  nature  of  activity  occurring  in  a site, 
and  apply  appropriate  weighting  to  tune  the  results  to  what  the  learners  would  be  better  interested  in. 

Attachment/V alue  Add 

The  final  aspect  we  attempted  to  measure  was  the  sense  of  “attachment”  to  the  recommendation,  or  would 
the  learners  feel  a sense  of  loss  if  they  no  longer  were  given  recommendations  of  discussion  activity.  The  majority 
of  students  indicated,  they  would  not  feel  a sense  of  loss  if  they  did  not  have  the  recommendations,  which  is  not 
surprising  given  the  fact  many  did  not  find  the  recommendations  useful  in  the  first  place  (see  Figure  4). 
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Figure  4:  Frequency  distribution  of  respondents  to  the  combined  survey  questions  regarding  “Attachment” 


Attachment  (Combined  Q1,Q9) 


1 2 3 4 5 6 7 


Response 


Conclusion 

Despite  finding  our  recommendation  system  easy  to  use,  the  majority  of  participants  in  our  study  did  not 
find  the  recommendations  provided  all  that  interesting.  Thus,  additional  research  will  be  required  to  better 
understand  what  types  of  discussion  board  activity  the  students  find  most  useful.  If  we  can  ascertain  the  “useful”  or 
more  valuable  discussion  board  activity  we  may  be  able  to  train  our  recommendation  system  to  weight  this  activity 
more,  and  hopefully  provide  better  recommendations  to  the  students. 
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Enhancing  user  participation  in  an  art  gallery  space  through  the 
development  of  a user  generated  wiki 
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Abstract 

A problem  exists  today  around  art  galleries.  An  galleries  that  do  not  keep  permanent  and  publicly 
accessible  collections  or  records  of  the  artwork  they  display  do  not  provide  opportunities  for  artists  and  viewers  to 
revisit  artwork  that  has  been  removed  from  the  gallery.  The  purpose  of  this  study  is  to  test  the  usability  of  a wiki 
platform  for  a university  art  gallery  that  allows  past  and  present  exhibiting  student-artists  and  any  other  interested 
parties  to  create  professional  profiles  and  alternative  exhibition  spaces  on-line  in  order  to  engage  with,  deepen, 
reflect  upon,  share  among,  translate  between  and  expand  their  experiences  with  and  understandings  of  art. 

Introduction 

The  use  of  the  Internet  by  people  of  all  ages  around  the  world  (Lenhart  2009;  Sutter  2009)  to  create,  remix, 
and  share  media  content  (Lenhart  & Madden,  2005)  through  engagement  in  “participatory  cultures”  (Jenkins,  2006) 
is  rapidly  increasing  (Zuckerberg,  2009).  According  to  Lu  (2008),  teaching  professionals  in  the  21st  century  “need  to 
think  about  how  to  engage  and  motivate  students  into  using  their  favorite  and  familiar  digital  media  as  effective 
learning  technologies”  (p.  48). 

At  the  Pennsylvania  State  University  (2007),  over  forty-thousand  students  are  currently  enrolled  at  the 
University  Park  campus,  just  one  branch  of  the  larger  University  system.  According  to  a survey  conducted  by  the 
Educational  Technology  Services  at  Penn  State,  83%  of  Penn  State  students  are  in  the  Facebook  social  network, 

93%  own  cell  phones,  and  nearly  90%  own  MP3  players  (Camplese,  2008).  The  every-day  lives  of  these  college 
students  are  becoming  increasingly  interwoven  with  the  use  of  contemporary  popular  technologies. 

As  a teaching  and  learning  space  at  Penn  State,  the  Edwin  W.  Zoller  Gallery  is,  centrally,  a space  for 
students  to  push  the  boundaries  of  how  they  create  and  think  about  art.  Each  academic  semester,  the  Zoller  Gallery 
hosts  as  many  as  seven  public  exhibitions  of  student  art  work,  and  it  is  expected  that  entire  classes  of  students  will 
come  into  the  gallery  with  their  instructors  at  any  given  time  to  talk  about  the  artwork  on  display.  However,  because 
the  Zoller  Gallery  keeps  no  permanent  collection  of  the  artwork  it  displays,  unlike  a museum,  students  have  no 
means  to  revisit  an  exhibition  of  artwork  at  the  gallery  after  one  has  been  removed  to  make  way  for  the  next. 

Finding  this  inability  of  students  to  revisit  prior  learning  experiences  problematic,  this  design-based 
research  study  seeks  to  provide  contextually  relevant  and  meaningful  opportunities  for  more  media-rich  and 
dynamically-connected  art  educational  experiences  within  and  beyond  the  Zoller  Gallery.  By  providing  ways  for 
students  to  engage  with  social  media,  both  inside  and  outside  of  the  gallery,  about  the  art,  artists,  and  events 
happening  there,  the  gallery  becomes  more  culturally  responsive  and  connected  to  contemporary  student  culture. 
Through  the  adaptation  of  social  media  to  enhance  and  extend  the  opportunities  for  teaching  and  learning  about  and 
through  art,  the  Zoller  Gallery  can  offer  students  an  alternative  to  the  limitations  of  their  current  gallery  experiences 
by  providing  them  with  an  emergent  multi-media  exhibition  and  conversation  platform  to  germinate. 

Study 

The  purpose  of  this  study  is  to  test  the  usability  of  a wiki  platform  associated  with,  but  not  bound  to  or 
restricted  by,  the  Zoller  Gallery.  This  wiki  allows  past  and  present  exhibiting  student-artists  and  any  other  interested 
parties  to  create  personal  (artist)  profiles  and  alternative  exhibition  spaces  on-line  in  order  to  engage  with,  deepen, 
reflect  upon,  share  among,  translate  between  and  expand  their  experiences  with  and  understandings  of 
art.  Following  the  creation  of  an  artist  profile,  artists  upload  an  artwork,  provide  descriptions  of  the  work,  and  allow 
other  users  of  the  wiki  platform  to  provide  comments  and  feedback  on  a whiteboard  associated  with  that 
artwork.  Figure  1 provides  a screenshot  of  this  discussion  and  feedback  feature  of  the  wiki  platform. 
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Figure  1.  Art  gallery  wiki  discuss  page.  This  figure  provides  an  example  of  the  whiteboard  commenting  feature  for 
the  artwork  entitled  Catching  a Bus  at  PSU. 

A qualitative  usability  test  method  was  used  to  conduct  this  study.  We  followed  Goto  and  Cotier’s  four- 
step  process  in  setting  up  our  usability  test.  Goto  and  Cotier  recommend  planning  and  prepping,  finding  participants, 
conducting  the  study,  and  finally,  analyzing  data  and  making  recommendations  based  on  the  results  (2005).  We 
followed  their  guidelines  and  used  their  Likert-type  post  session  survey  items  as  part  of  our  user  post-test  survey. 

The  goals  of  our  usability  test  were  to  provide  specific  feedback  on  common  tasks  within  the  system  and  determine 
expected  performance  on  the  current  site  and  identify  serious  problems  prior  to  the  next  phase  of  production.  By 
conducting  a usability  test,  we  wanted  to  answer  the  following  questions:  Are  the  basic  tasks  intuitive  for  a new  user? 
Are  the  menu  options  and  features  intuitive  for  a new  user? 

Method 

Our  study  consisted  of  four  graduate  students  that  regularly  engage  and  participate  in  an  art  gallery 
space.  We  gathered  two  types  of  background  information.  First,  we  asked  how  many  hours  a participant  spent 
online  per  day,  and  second,  we  asked  our  participants  to  complete  a pre-test  power  user  scale  survey  to  determine 
how  “tech-savvy”  a person  is.  Marathe  et  al.  created  the  power  user  scale  for  classification  purposes  when 
conducting  media  effect  studies  (2007).  The  power  user  scale  consists  of  nine  Likert-type  items  and  are  provided  in 
Appendix  A. 

Each  session  lasted  approximately  30  minutes.  During  the  usability  test,  each  subject  was  greeted  and  then 
oriented  to  the  design  by  a test  monitor.  Subjects  were  given  four  tasks  to  complete  while  the  test  monitors 
observed.  Participants  were  asked  to  think-aloud,  i.e.  talk  out  their  thought  process,  while  they  completed  the  four 
tasks.  The  four  tasks  are  listed  in  Appendix  B.  Following  the  tasks,  a post-test  survey  was  given  in  addition  to  a 
debriefing  on  the  significance  of  the  study. 


Results 

The  number  of  hours  spent  online  was  roughly  equivalent  across  subjects,  showing  that  our  participants  did 
have  experience  in  online  web  applications  with  an  average  of  5.72  hours  spent  online  per  day.  High  power  users 
consisted  of  participants  who  scored  high  across  all  items  on  the  power  user  scale  and  low  power  users  scored 
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low.  We  found  that  two  of  our  participants  scored  high  (average  of  8.278)  and  two  of  our  participants  scored  low 
(average  of  4.61 1).  In  our  study,  the  power  user  scale  items  were  compared  using  a Multivariate  factor  analysis  to 
determine  that  the  scale  was  reliable  (Cronbach’s  alpha  of  0.8033). 

When  comparing  our  high  and  low  power  users  across  post-test  survey  results,  we  found  that  the  high 
power  (tech-savvy)  users  consistently  scored  higher  across  all  items.  These  items  are  listed  in  Table  1.  Each  item  is 
a score  out  of  5 (l=strong  disagree;  5=strongly  agree). 


Low  Power  Users 

High  Power  Users 

Easy  to  complete  tasks 

4 

5 

Clear  Navigation 

4.5 

5 

Pleasing  Look  and  Feel 

3.5 

5 

Relevance  of  images  to  content 

4 

5 

Relevance  of  content  (text) 

4 

5 

Easy  to  use 

4 

5 

Clear  page  layout 

4 

5 

Inviting  to  use 

4 

4.5 

Clear  labeling  of  links 

4.5 

4.5 

Recommend  a friend  to  use 

3.5 

5 

Table  1.  Low  vs.  High  Power  Users.  This  table  provides  the  score  for  each  group  of  participants  on  the  post-test 
survey. 


Additionally,  we  provided  three  post-test  survey  open-ended  questions.  The  participants  that  we  classified 
as  low  power  users  found  that  the  best  features  of  the  wiki  platform  were  the  numerous  ways  to  post  and  transmit 
ideas.  The  high  power  users  thought  the  best  feature  of  the  wiki  platform  was  the  ease  of  use.  The  low  power  users 
found  that  there  was  a lack  of  clarity  with  certain  tasks  (e.g.  the  discussion  board)  while  the  high  power  users  were 
pleased  with  the  system  and  listed  the  generic  color  scheme  as  the  only  complaint.  When  asked  what  would  you  do 
to  enhance  the  site,  one  low  power  user  participant  called  for  more  simplification  of  the  features  in  the  wiki 
platform.  One  of  the  high  power  user  participants  asked  for  a tagging  system  to  categorize  the  types/mediums  of 
artwork. 

During  the  think-aloud  process,  subjects  pointed  out  a few  important  concerns  that  were  not  addressed  in 
our  post-survey  results.  Below  we  provide  what  we  found  to  be  the  most  important  concerns: 

> Lack  of  information  on  the  goals  and  outcomes  of  the  site 

> Lack  of  an  “update”  button  after  editing  text  (current  system  automatically  updates) 

> Problem  finding  the  login  area/information  (current  system  has  login  on  top  right) 

> Majority  of  feedback  will  be  text-based,  so  make  the  whiteboard  discussion  an  advanced  feature 

> Aesthetics  of  the  site  should  be  changed;  too  much  of  a template 

> Possibility  to  use  system  with  another  school  to  collaborate  and  comment  on  other  University 
student/artists’  artwork 

> Lack  of  ability  to  see  text  descriptions  when  you  hover  over  buttons 

> Lack  of  prompts  of  what  to  write  in  description  boxes 

> Explanation  needed  for  whiteboard  discussion  board 

> Lack  of  ability  to  make  artwork  full-screen 

Discussion 
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Our  results  show  that  the  participants  that  are  considered  high  power  users  and  “tech-savvy”  found  the  site 
to  be  easy,  clear,  inviting,  and  relevant.  In  other  words,  the  wiki  platform  architecture  is  appropriate  and  the  system 
serves  the  function  for  which  it  was  created.  This  is  an  important  finding  as  we  move  forward  to  our  next 
prototype.  For  those  participants  that  are  considered  less  “tech-savvy”,  the  findings  are  still  encouraging.  These 
participants  also  agreed,  although  not  as  strongly,  with  the  ease  and  clarity  of  the  site.  These  results  provide  us  with 
a positive  response  to  our  questions  regarding  the  intuitive  nature  of  the  tasks  and  menu  options. 

The  open-ended  questions  and  think-aloud  responses  provided  us  with  very  valuable  feedback  and 
suggestions  for  improvement.  A recurring  theme  across  participants  was  the  need  for  a better  aesthetic  and  color 
scheme.  The  suggestion  to  add  an  artistic  background  is  something  that  our  design  team  will  factor  into  our  next 
prototype.  We  did  get  a positive  response  with  the  simplicity  of  the  wiki  platform  and  therefore,  we  are  going  to 
ensure  that  adding  artistic  elements  do  not  interfere  with  the  simplicity  of  the  function  and  navigability. 

An  interesting  find  was  the  participant  confusion  and  realization  that  text  edits  were  automatically  synced 
across  the  system.  Our  participants  are  used  to  having  submit  buttons  upon  completing  a task  and  thus,  thought  that 
this  feature  was  needed.  This  finding  is  intriguing  because  it  suggests  that  we,  as  consumers  of  web  media,  are 
trained  by  web  designers  to  locate  buttons  when  editing  text.  In  order  to  accommodate  this  disconnect,  our  design 
team  will  add  a text  rollover  effect  that  informs  the  user  of  “editing  text”  and/or  “text  submitted"  during  the  edit 
process,  thus,  eliminating  the  need  for  a button. 

A non-design  suggestion  from  the  think-aloud  responses  provided  us  with  an  opportunity  to  reflect  on  our 
goals  and  outcomes  and  realize  the  potential  and  excitement  that  this  system  could  generate.  The  suggestion  was 
using  the  wiki  platform  with  another  school  to  collaborate  and  comment  on  other  University  student/artists’  artwork. 
Although  we  are  a few  prototypes  away  from  implementing  this  platform  on  that  scale,  this  suggestion  helps  us 
refocus  our  design  principles  and  rethink  our  social  problem.  By  increasing  collaboration  among  Universities,  our 
wiki  platform  could  serve  as  a solution  to  the  problem  of  removed  art  education;  removed,  as  in  removed  from 
society. 


Implications 

In  the  next  phase  of  our  work,  we  will  add  in  many  of  the  suggestions  from  this  study.  We  plan  to  move 
forward  with  the  prototype  and  increase  the  functionality  of  the  wiki,  including  the  amount  of  tasks  that  a user  can 
perform  without  sacrificing  its  simplicity,  clarity,  and  ease  of  use.  Through  this  and  subsequent  usability  studies,  we 
hope  to  create  a communication  platform  that  is  both  highly  usable  by,  and  useful  for,  student-artists  who  want  to 
engage  in  enhanced  and  extended  educational  experiences  around  artwork  exhibited  at  the  Zoller  Gallery. 
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APPENDIX  A 
Power  user  scale 

On  a scale  of  1 to  10  ( 1=LEAST  AGREE;  10=MOST  AGREE),  do  you: 

1 . Think  most  technological  gadgets  are  complicated  to  use. 

2.  Love  exploring  all  the  features  that  any  technological  gadget  has  to  offer. 

3.  Prefer  to  ask  friends  how  to  use  any  new  technological  gadget  instead  of  trying  to  figure  it  out 
myself. 

4.  In  interfaces  that  you  are  familiar  with,  do  you  get  frustrated  each  time  you  have  to  go  through 
basic  steps  designed  for  new  users. 

5.  Like  to  challenge  yourself  in  figuring  out  how  to  use  any  new  technology. 

6.  A little  bit  of  intuition  is  all  that  is  needed  to  figure  out  how  to  use  any  new  technology. 

7.  Many  of  your  friends  come  to  you  to  get  help  related  to  technological  gadgets. 

8.  Find  yourself  using  keyboard  shortcuts  on  the  computer. 

9.  Make  use  of  the  ‘Help'  feature  available  in  any  software. 

APPENDIX  B 
Task  Information 

1 . Your  first  task  is  to  enter  a description  for  your  artwork  Catching  a Bus  at  PSU.  Update  the  description 
with  a few  phrases.  If  you  are  prompted  to  log  in,  use  the  username  mike  and  password  mike.  Once  you 
have  accomplished  this  task,  briefly  discuss  the  positive  and  negative  aspects  of  the  system  during  this 
task. 

2.  Your  second  task  is  to  change  the  Artist  Biography  Information  for  your  video  by  going  to  My 
Profile.  Once  you  add  a few  phrases,  go  back  to  the  initial  artwork  page  ART  - ABOUT  tabs  and 
comment  on  what  is  different.  Briefly  discuss  the  positive  and  negative  aspects  of  the  system  during 
this  task. 

3.  Your  third  task  is  to  provide  feedback  for  the  artwork  on  the  ART  - discuss  tab.  If  you  haven't  already, 
you  may  watch  the  video  to  help  with  this  process.  Explore  the  function  of  the  discussion  board  and  see 
what  you  can  do.  Comment  on  your  experiences  including  suggestions  for  improvement. 

4.  Your  final  task  is  to  add  a question  to  the  Q+A  discussion  board.  Your  question  can  be  about  the  art 
video  or  a general  question.  Once  you  have  accomplished  this  task,  briefly  discuss  the  positive  and 
negative  aspects  of  the  system  during  this  task. 
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Abstract 

In  an  attempt  to  address  the  limited  academic  resources  many  nursing  and  allied  health  students  face,  this 
project  engages  healthcare  students  with  mentors  via  a learning  management  system.  This  paper  describes  a novel 
preload  teaching  technique  and  discusses  the  use  of  online  social  networks  for  tutoring  and  mentoring  of  science 
students.  This  article  closes  with  recommendations  that  can  be  implemented  when  academic  practitioners  employ 
technology  to  facilitate  mentoring.  Further,  this  paper  suggests  how  the  social  networking  aspects  of  learning 
management  systems  can  support  student's  academic  goals  outside  of  the  traditional  classroom  setting. 

Introduction 

Traditional  educational  institutions  often  have  limited  resources  for  healthcare  students,  resulting  in 
student-faculty  ratios  that  are  prohibitively  large  and  may  inhibit  learning  (Biemiller,  2009).  This  paper  describes  a 
project  that  engages  nursing  and  healthcare  technology  students  with  academic  tutors  before  their  science  courses 
begin  and  prepares  them  for  learning  by  addressing  the  most  challenging  topics  in  the  subject  area.  The  project  also 
implements  a mentoring  program,  where  students  of  anatomy  and  physiology  courses,  and  healthcare  professionals 
are  connected  via  a learning  management  system  (LMS)  and  online  social  networks  during  the  academic  term.  The 
learning  management  system  (LMS)  facilitates  the  use  of  peripheral  participation  (Lave  and  Wagner,  1990)  and 
situated  cognition  (Brown  and  Collins,  1989)  by  mentors  and  tutors  outside  of  the  traditional  classroom  to  achieve 
the  student’s  academic  goals. 


Background 

The  healthcare  community  is  currently  facing  a shortage  of  qualified  nurses  and  technical  professionals. 
This  is  having  detrimental  impacts  on  both  patient  care  and  financial  outcomes.  The  problem  is  projected  to  get 
much  worse  between  now  and  2020  as  existing  professionals  enter  retirement  ( AACN,  2008;  Buerhaus,  et  al.,  2009). 
This  project  is  designed  to  prepare  current  clinical  and  non-clinical  health  care  workers  (i.e.,  licensed  vocational 
nurses  (LVNs),  phone  operators,  billing  specialists,  etc.)  and  assist/sponsor  them  in  getting  their  two  year  Registered 
Nurse  (RN)  degree  or  Allied  Health  Technology  degree.  These  degree  programs  typically  include  one  year  of 
prerequisite  science  courses  before  degree  courses  begin.  A significant  problem  arises  because  many  potential 
students  find  the  demands  of  the  prerequisite  courses  and  subsequent  health  science  courses  to  be  extremely 
challenging  given  their  adult  learner  status  (i.e.,  having  full-  or  part-time  jobs,  family  responsibilities,  etc.)  (Okezie, 
2009).  Additionally,  allied  health  students  often  find  the  technical  language  and  culture  of  scientific  inquiry  to  be  a 
new  challenge  that  adds  significantly  to  the  difficulty  of  a course  (Rogers  and  Cox,  2009).  Analysis  shows  that  a 
major  factor  preventing  students  from  enrolling  in  allied  health  and  nursing  programs  is  failure  to  complete 
prerequisite  science  courses  (Blackman  and  Hall,  2009).  Failure  to  complete  prerequisite  science  courses  means 
lower  numbers  of  qualified  applicants  for  admission.  This  is  having  a detrimental  effect  on  the  current  workforce  in 
two  ways.  First,  failure  to  enroll  in  training  programs  is  accentuating  the  current  shortage  caused  by  the  increase  in 
demand  for  nurses  and  allied  health  professionals.  Second,  the  future  of  health  care  staffing  is  further  challenged  as 
normal  attrition  of  current  practitioners  continues,  there  is  limited  replacement  with  adequately  trained  professionals. 

Currently,  several  large  entities  in  the  healthcare  industry  with  a vested  interest  in  promotion  of  a new 
generation  of  nurses  and  allied  health  practitioners  have  collaborated  to  initiate  novel  training  programs. 

Specifically,  Kaiser  Permanente  (HMO),  Service  Employees  International  Union  (SEIU),  United  Health  Workers  - 
West  (UHW),  and  the  SEIU  UHW-West  and  Joint  Employer  Education  Fund  are  approaching  this  problem  using 
technology,  tutoring,  and  mentoring. 
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Definitions 

In  order  to  facilitate  discussion,  this  section  defines  a number  of  terms  that  have  specific  meanings  within  the 
context  of  this  paper  that  may  be  slightly  different  than  academic  tradition. 

• Adult  Learner  - An  adult  learner  is  a mature  student  enrolled  a formal  academic  program  leading  towards  a 
degree  or  professional  certification  that  has  distinct  motivations  and  responsibilities  outside  of  the  academic 
setting  that  sets  them  apart  from  a classic  academic  student. 

• Learning  Management  System  - A learning  management  system  is  a comprehensive  online  server  and 
database  that  serves  as  a coordinated  hub  for  instructors  and  students  to  communicate  and  use  learning 
tools. 

• Short  Message  Service  (SMS)  - A text  messaging  service  that  allows  communication  between  cell  phones, 
email  and  LMS. 

• Tutor  - A tutor  is  an  educator  and  subject  matter  expert  that  is  assigned  to  a student  outside  of  class  time  to 
assist  them  with  their  learning  goals.  A tutor  may  also  be  a mentor. 

• Mentor  - A mentor  is  a subject  matter  expert  that  is  charged  with  providing  support,  context  and  guidance 
in  a specific  community  of  practice.  A mentor  may  also  act  as  a tutor. 

• Preload/frontload  - The  Preload/frontload  teaching  model  exposes  students  to  the  class  objectives  six 
weeks  before  the  class  starts  in  order  to  give  them  a greater  amount  of  time  to  achieve  competency. 

• Online  Social  Networking  - Online  Social  Networking  is  the  use  of  electronic  and  multimedia  materials  to 
facilitate  quick  and  effective  communication  between  groups  of  people. 

• Retention  - Retention  is  the  ability  to  keep  a person  involved  and  participating  in  a tracked  academic 
program  or  professional  job. 

• Allied  Health  Care  - Allied  health  care  is  defined  as  the  numerous  technical  and  clinical  professions  that 
act  as  the  backbone  of  the  medical  system.  Allied  health  care  is  distinct  from  nursing. 

• Clinical  Sciences  - Clinical  sciences  are  the  academic  and  applied  scientific  disciplines  (anatomy, 
physiology,  physics,  chemistry,  etc...)  that  are  the  basis  for  clinical  practice  in  a healthcare  setting. 

• Student  - A student  is  a person  enrolled  in  an  academic  or  clinical  training  program. 

• Non-clinical  health  care  workers  - Non-clinical  health  care  workers  are  people  that  participate  in  a clinical 
setting  and  have  direct  patient  contact,  but  do  not  administer  therapies  or  other  clinical  modalities. 

• Educator  - The  educator  is  the  professional  from  an  academic  institution  that  has  been  tasked  with  the 
responsibility  of  delivering  course  content  and  making  sure  that  the  student  has  achieved  their  academic 
and  clinical  competencies.  This  person  is  unique  from  the  mentor  or  tutor. 

Methods 

In  order  to  facilitate  the  nursing  and  allied  health  care  student’s  success  in  prerequisite  science  courses,  the 
SEIU-UHW  Ed  Fund  used  an  LMS  to  preload  students  in  order  to  connect/facilitate  mentors  and  tutors  with 
students.  Traditional  education  institutions  have  limited  resources  (Martinez,  2000)  for  science  students,  and  these 
resources  are  limited  to  remedial  face-  to  -face  tutoring.  Additionally,  class  sizes  are  very  large  and  instructor- 
student  ratios  are  prohibitively  small  (Biemiller,  2009).  A classic  example  of  this  type  of  class  is  anatomy  and 
physiology.  The  traditional  tutoring  model  most  often  takes  students  already  behind  or  struggling  and  tries  to  catch 
them  up  academically.  The  novel  preload/frontload  model  proposed  in  this  paper  is  designed  to  take  students,  before 
the  science  course  starts,  and  expose  them  to  the  core  objectives  and  competencies  addressed  in  the  class. 
Specifically,  this  preload  course  engages  anatomy  and  physiology  students  that  have  notified  their  instructors  or 
unions  of  their  intention  to  apply  for  a professional  nursing  or  allied  health  training  program. 

The  preload/frontload  model  is  a preliminary  short  course  of  four  to  six  weeks  duration  that  addresses  the 
most  challenging  topics  in  the  upcoming  subject  area  (anatomy  and  physiology).  This  "preload  class"  is 
accompanied  by  a follow-on  mentoring  program  where  students  are  paired  with  healthcare  professionals  or 
academic  instructors  from  other  institutions.  The  mentors  are  financially  compensated  for  their  participation. 

Two  major  factors  set  this  program  apart  from  other  tutoring  models.  The  first  difference  is  that  mentors 
are  connected  to  their  students  via  a standard  LMS  or  a social  networking  site.  Specifically,  this  project  offers  a 
Moodle  based  LMS  for  the  mentors  (Branzburg,  2009).  This  is  in  addition  to  any  LMS  that  may  be  employed  by  the 
academic  institution.  The  mentoring  employed  in  this  program  is  distinct  from  the  preload  tutoring.  The  mentors 
may  also  perform  tutoring  services,  however  what  sets  mentors  apart  from  tutors  is  that  mentors  provide  both  real 
world  context  to  the  course  objectives  and  provide  an  opportunity  for  the  students  to  initiate  situated  cognition  and 
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legitimate  peripheral  participation  in  the  nursing  and  allied  health  care  fields  before  they  are  in  clinical  training.  This 
role  stands  in  contrast  to  tutors  that  only  give  clarification  to  classroom  objectives  and  are  often  times  little  more 
than  advanced  upper  classmen  themselves. 

Preload  classes  are  delivered  in  a classroom.  Audio  is  recorded  and  recordings  are  provided  via  podcast 
technology  from  a central  campus  location.  Additionally,  these  preload  lectures  are  posted  on  the  course  LMS, 
allowing  for  remote  viewing  in  an  online  space  that  supports  interaction  with  mentors  and  tutors.  Students  are 
expected  to  attend  at  least  50%  of  the  preload  classes  in  person  (in  the  classroom),  but  some  classes  may  be 
"attended"  through  online  asynchronous  participation  using  the  prerecorded  lectures. 

Furthermore,  students  are  given  electronic  and  paper  media  materials  (quick  reference  guides)  to  facilitate 
conceptual  understanding  of  preload  topics.  The  media  (interactive  test  banks,  online  flash  cards,  etc.)  along  with  the 
recorded  preload  lectures  are  also  available  to  the  students  and  mentors  via  the  LMS  during  their  anatomy  and 
physiology  course.  Although  not  always  the  case,  the  instructor  of  the  preload  course  is  encouraged  to  take  on  the 
role  of  mentor,  modifying  and  using  the  preloaded  online  resources  to  assist  the  students  as  they  complete  the 
science  course.  Additionally,  this  “tutor  becomes  mentor”  transition  allows  students  to  centrally  locate  and  interact 
with  the  preload  instructor  and  other  participating  students  during  the  science  course.  The  use  of  this  technology  will 
be  continuously  made  available  to  the  students  and  is  expected  to  persist  through  the  student's  academic  career  while 
they  are  in  their  professional  nursing  and  allied  health  programs.  Specifically,  the  LMS  tools  like  blogs,  wikis,  and 
online  chat  functions  allow  the  LMS  to  becoming  a social  network  and  resource  that  the  student  can  return  to  and 
utilize  with  other  courses.  Subjective  reports  and  survey  results  discussed  in  this  paper  indicate  that  the  cohort  and 
mentor  relationships  developed  during  both  stages  will  persist  through  the  student's  professional  studies.  Because 
the  literature  shows  that  cohort  training  increases  retention  (Imel,  2009),  the  use  of  the  LMS  and  above  mentoring 
strategies  are  expected  to  increase  student  enrollment  and  reduce  attrition.  Additionally,  because  situated  cognition 
and  limited  peripheral  participation  increase  student  achievement  (O'Donnell  and  Tobbell  2009;  Hung,  2009),  the 
ability  of  the  mentor  to  express  how  current  academic  subjects  relate  to  their  practice  and  the  student's  future 
professional  practice  can  likewise  increase  student  success. 

Students  that  participated  in  the  mentoring/tutoring  program  either  volunteered  through  their  union  or  were 
mandated  by  their  employer  to  participate. 

Students  in  the  mentoring/tutoring  program  were  voluntarily  asked  to  participate  in  a survey.  In  order  to 
gain  the  comparative  perspective  of  students  not  in  the  mentoring  program,  the  mentors  and  tutors  also  invited 
students  from  similar  classes  that  were  not  enrolled  in  the  mentoring  program  to  participate  in  the  survey.  The 
decision  to  include  non-mentored  students  in  the  survey  was  based  on  their  stated  intention  to  enroll  in  a 
professional  or  technical  healthcare  training  program. 

Thirty  nine  students  responded  to  the  survey.  Respondents  attended  several  local  community  colleges  and 
technical  schools. 

The  survey  was  conducted  anonymously  and  was  designed  to  elicit  information  about  participant  history, 
utilization  data  and  personal  satisfaction.  A Likert  scale  was  employed  to  collect  subjective  student  responses.  No 
student  demographic  data  (age,  name,  etc.)  or  student  achievement  data  (GPA)  was  recorded. 

Results 

The  data  has  been  organized  into  seven  areas  of  interest.  These  areas  of  interest  are  based  on  the  compiled 
information  collected  by  the  survey.  Each  of  the  seven  areas  of  interest  is  summarized  in  the  following  analysis.  The 
results  are  further  discussed  in  the  discussion  section  later  in  this  paper. 

1 . When  questioned  about  their  previous  behavior  and  utilization,  students  reported  that  at  some  time  in  their  adult 
academic  career: 

• 92.3%  of  respondents  had  used  a free  online  community  or  social  network  to  facilitate  learning 
(activities  included  file  sharing,  short  message  services,  uploading  of  audio  lectures  and  lecture 
transcripts,  posting  question  blogs,  class  notes,  and  hyper-linking  to  graphics  and  other  internet 
resources) 

• 56.4%  of  respondents  had  used  Blackboard 

• 25.6%  of  respondents  had  used  Moodle 

• 20.5%  of  respondents  had  used  Evolve 

• 10%  of  respondents  had  used  some  other  commercial  LMS 

Analysis:  Free  online  social  networking  sites  are  being  used  extensively  by  students  to  facilitate  learning  in 
traditional  academic  settings. 
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2.  Students  reported  that  of  the  above  learning  management  tools  that  they  had  used  in  the  past,  (including  social 
networking  sites): 

• 66.7%  had  stated  that  the  resource  was  provided  by  the  school 

• 56.4%  had  stated  that  the  resource  was  provided  by  the  instructor  (privately  hosted  website,  or 

moderated  on  a social  network)  outside  of  the  schools 

• 76.9%  had  used  a resource  provided  by  another  student 

• 15.4%  had  implemented  the  resource  for  themselves  and  their  friends 

Analysis:  If  the  school  or  instructor  is  not  providing  a forum  for  internet  discussion  or  exchange  of  resources, 
students  will  use  online  social  networks  to  do  so  anyway. 

3.  With  regards  to  students  currently  enrolled  in  a science  course  for  the  purpose  of  fulfilling  a prerequisite  in 
order  to  gain  admission  into  a professional  or  technical  health  care  program  (as  of  the  time  of  the  survey): 

• 12.8%  were  assigned  a tutor  by  their  employer 

• 7.7%  had  chosen  to  voluntarily  participate  with  a tutor/mentor 

• 30.8%  did  not  use  a tutor  or  mentor  or  study  group 

• 53.8%  formed  their  own  study  group 

Analysis:  Because  the  sum  of  the  number  of  students  reporting  their  participation  status  is  greater  than  100%,  a 
small  percentage  of  students  (5%)  that  had  tutors  and  mentors  must  also  have  been  involved  in  self  formed 
study  groups.  Also,  students  more  often  form  their  own  study  group  than  voluntarily  participate  with  a mentor 
or  tutor. 

4.  With  regards  to  students  previously  enrolled  in  clinical  science  courses  for  the  purpose  of  fulfilling  a 
prerequisite  so  as  to  gain  admission  into  a professional  or  technical  health  care  program  (as  of  the  time  of  the 
survey): 

• 17.9%  had  been  assigned  a tutor  by  their  employer 

• 15.4%  had  chosen  to  voluntarily  participate  with  a tutor/mentor 

• 28.2%  had  not  used  a tutor  or  mentor  or  study  group 

• 56.4%  formed  their  own  study  group 

Analysis:  The  rates  of  students  that  choose  to  not  use  a mentor/tutor/study  group  and  the  rates  of  students  that 
form  their  own  study  group  appears  to  be  consistent  between  current  class  and  classes  taken  in  the  past. 

5.  The  respondents  stated  the  following  with  regards  to  tutoring  or  mentoring  participation  at  any  time  in  the 
past: 

• 76.9%  of  students  that  had  participated  in  a tutoring  or  mentoring  program  either  this  term  or  in 
past  terms  agreed  that  they  would  participate  in  a tutoring/mentoring  program  again  if  available. 

• 100%  of  students  that  participated  in  a tutoring  or  mentoring  program  either  this  term  or  in  past 
terms  stated  that  their  participation  in  the  program  helped  facilitate  their  academic  goals 

• 84.6%  of  students  that  participated  in  a tutoring  or  mentoring  program  either  this  term  or  in  past 
terms  agreed  that  the  use  of  learning  management  tools  and  social  networks  helped  facilitate  their 
group  tutoring/mentoring. 

• 51.3%  of  students  that  participated  in  a tutoring  or  mentoring  program  agreed  that  they  would 
likely  stay  in  contact  with  their  peers.  This  is  contrasted  with  only  33%  of  students  in  a tutoring  or 
mentoring  program  without  an  LMS  agreeing  that  they  would  stay  in  contact  with  their  peers.  (No 
conclusion  maybe  made  from  this  data  alone  as  the  students  that  had  tutors  but  no  LMS 
availability  are  most  likely  students  in  one  to  one  tutorial  sessions,  and  did  not  have  other  students 
in  their  tutorial  sessions  to  stay  in  contact  with.) 

Analysis:  Students  that  participate  in  mentoring  and  tutoring  (not  accounting  for  voluntary  vs.  mandated 
participation)  have  a high  degree  of  subjective  satisfaction  with  their  tutoring/mentoring  program  and  feel  that 
using  an  LMS  to  facilitate  the  program  was  beneficial. 

6.  The  respondents  stated  the  following  with  regards  to  study  group  participation  at  any  time  in  the  past: 

• 72.2%  of  students  that  had  participated  in  a study  group  either  this  term  or  in  past  terms  agreed 
that  they  would  participate  in  a study  group  again  if  available. 

• 81.8%  of  students  that  participated  in  a study  group  program  either  this  term  or  in  past  terms 
stated  that  their  participation  in  the  program  helped  facilitate  their  academic  goals 

• 90.9%  of  students  that  participated  in  a study  group  with  an  LMS  either  this  term  or  in  past  terms 
agreed  that  the  use  of  learning  management  tools  (of  any  sort)  helped  facilitate  their  group 
tutoring/mentoring . 
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• 68.2%  of  students  that  participated  in  a study  group  with  an  LMS  agreed  that  they  would  likely 
stay  in  contact  with  their  peers.  As  opposed  to  27%  of  students  in  a study  group  without  an  LMS 
agreeing  that  they  would  stay  in  contact  with  their  peers. 

Analysis:  Students  that  participate  in  study  groups  have  a high  degree  of  subjective  satisfaction  with  their 
study  groups  and  using  an  LMS  to  facilitate  their  program  further  adds  to  this  subjective  satisfaction.  Study 
group  students  appear  to  find  greater  subjective  value  in  the  use  of  an  LMS  than  do  tutored  students.  Study 
groups  have  a lower  reported  rate  of  willingness  to  participate  again  in  a future  class  than  do  mentoring/tutoring 
groups.  However,  study  groups  that  use  an  LMS  or  social  network  have  a higher  reported  chance  of  staying  in 
contact  with  fellow  participants  than  do  the  mentored  students  that  use  an  LMS  or  social  network. 

7.  With  regards  to  learning  management  systems,  students  were  asked  about  which  resources  they  used  in  their 
currently  enrolled  class,  which  resources  they  were  likely  to  use  in  the  future,  and  which  resources  were  most 
effective  in  helping  them  achieve  their  learning  goals.  The  following  resources  were  discussed: 

• Social  networking  (blog,  group  email,  Twitter)  to  communicate  with  students 

• Social  networking  (blog,  group  email,  Twitter)  to  communicate  with  instructors 

• Grade  tracking  to  monitor  class  performance 

• Database  access  to  share  materials  (student  notes,  graphics  etc.)  with  other  students 

• Online  quizzes/practice  tests 

• Prerecorded  lectures 

• Instructors  notes 

• Interactive  modules  and  tutorials 

• Electronic  submission  of  assignments 

Analysis:  In  general  students  responded  that  all  of  the  above  resources  were  both  helpful  and  that  they  would 
use  them  again  if  provided.  It  is  interesting  to  note  that  student  responses  indicated  that  the  majority  of  students 
(64%)  found  interactive  Flash  movies  very  valuable  resource  with  regards  to  helping  them  achieve  their 
academic  goals,  however  only  51.2%  the  students  stated  that  they  used  them  only  occasionally  when  available 
even  though  89.4%  of  students  reported  that  they  would  likely  choose  to  use  them  in  the  future.  In  contrast, 
fewer  students  (48.7%)  reported  that  they  found  practice  quizzes  more  helpful  than  they  did  the  interactive 
Flash  tutorials  in  achieving  their  academic  goals.  However,  a high  percentage  of  students  (79.5%)  reported  that 
they  had  used  practice  quizzes  very  often  when  they  were  made  available.  It  is  also  intriguing  to  note  that  only 
23%  of  students  reported  using  prerecorded  lectures  when  available,  but  58.9%  of  students  reported  that 
prerecorded  lectures  were  very  helpful  in  helping  them  achieve  their  goals,  and  64.1%  said  they  would  use 
prerecorded  lectures  again  if  made  available.  Conversely  79.5%  of  students  said  that  they  would  use  the  grade 
tracking  functions  again  and  71.5%  of  students  said  that  they  would  use  the  online  file  sharing  ability  again,  but 
that  28.9%  and  38.5%  of  students  respectively  stated  that  they  felt  grade  tracking  and  file  sharing  were  useful  in 
achieving  their  academic  goals. 


Discussion 

Because  of  the  high  utilization  rates  of  student  reported  data,  students  are  most  likely  using  social  networks 
even  when  an  LMS  is  made  available  to  them.  Because  15%  of  students  are  reporting  that  they  are  creating  their 
own  online  resources,  but  that  75%  are  utilizing  online  resources  supplied  by  other  students.  Whether  this  is  because 
a few  students  have  the  initiative  and  skill  to  setup  and  manage  social  networks  and  collaborative  file  sharing  sites, 
or  it  is  the  case  that  the  first  students  in  a class  to  create  the  resource  share  it  with  other  their  classmates  remains 
unknown.  In  either  case,  it  could  be  very  valuable  to  instructors  and  institutions  that  are  less  technologically  savvy 
to  model  the  practices  of  Iowa  State  University  (Beisser,  2000).  Beisser  suggest  that  academic  organizations  identify 
technologically  proficient  students  and  co-opt  them  into  helping  the  instructors  manage  electronic  resources  or  act  as 
tutors. 

It  also  remains  unexplained  why  the  data  here  shows  that  students  are  far  more  likely  to  form  study  groups 
than  use  tutors  or  mentors.  Even  though  most  colleges  provide  free  tutoring  centers  it  might  be  the  case  that 
accessibility  and  availability  with  regards  to  student  schedules  is  a factor.  It  is  also  plausible  that  while  tutors  and 
mentors  are  helpful,  having  a study  group  that  is  focused  on  the  specific  class  in  question  and  that  is  available  for 
project  collaboration  is  a big  draw.  This  theory,  that  increased  schedule  flexibility  and  collaborative  availability  are 
attractive  features  to  students,  can  also  be  applied  to  why  students  use  social  networks  at  such  high  rates  even  when 
an  LMS  is  available.  This  hypothesis  is  further  supported  by  the  work  of  Ciani  et  al.  which  shows  that  self 
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deterministic  learning  strategies  (students  creating  their  own  study  cohorts  as  opposed  to  being  assigned  a cohort) 
positively  impact  motivation,  retention,  and  subsequently  increases  achievement. 

It  should  be  noted  that  the  reported  data  comparing  the  use  of  tutoring/mentoring  and  student  formed  study 
groups  are  intertwined  with  the  novel  preload  approach  used  in  this  study,  and  that  it  is  difficult  to  make 
comparative  analysis  between  tutoring/mentoring  and  study  groups  and  the  use  of  academic  LMS.  Traditionally,  the 
academic  LMS  is  the  purview  of  classroom  academics  only.  However,  the  collected  subjective  data  suggests  that  the 
use  of  ether  an  online  social  network  or  an  LMS,  while  not  linked  to  a specific  academic  class,  can  be  very  useful  in 
facilitating  both  study  groups  and  tutoring/mentoring  programs.  This  paper  is  recommends  that  “drop  in”  college 
tutorial  centers  can  make  use  of  online  social  networking  to  reduce  tutoring  load  and  administrative  issues. 
Traditional  barriers  to  online  tutoring  include  excessive  amounts  of  time  addressing  repetitive  issues,  and  the 
volume  of  data  entry  by  keyboard  (Fox,  Seamus,  and  MacKeogh,  2009).  Reliance  on  student  formed  online  social 
networks  can  eliminate  or  reduce  these  barriers  by  allowing  students  to  post  responses  to  other  students  questions, 
and  allows  the  tutor  to  comment  and  correct  when  needed.  While  online  posts  and  Short  Message  Services  (SMS) 
will  not  take  the  place  of  traditional  face  to  face  tutoring,  the  use  of  an  LMS  for  tutoring  has  advantages.  First,  the 
availability  of  online  resources  increases  accessibility  for  people  that  may  not  be  able  to  otherwise  physically  get  to 
the  tutoring  center.  Second,  it  allows  more  users  at  the  same  time  to  access  the  tutor’s  information.  Lastly,  it  is 
common  for  tutors  to  encounter  the  same  questions  term  after  term.  With  online  resources,  previous  posts  can  be 
made  available  or  referenced  very  easily  to  for  frequently  asked  questions. 

We  recommend  further  research  be  done  to  compare  the  value  of  academic  LMSs  versus  the  value  of 
online  social  networks.  According  to  the  subjective  reported  data  of  this  paper,  the  most  utilized  aspect  of  a LMS  is 
the  ability  to  upload  and  download  shared  data  and  to  communicate  and  collaborate  with  other  students  and 
instructors.  It  should  be  noted  that  these  functions  can  be  managed  using  online  collaboration  resources  like  Google 
docs  and  Yahoo  groups.  Further,  even  the  subjectively  most  valuable  aspects  of  an  LMS  reported  by  students 
(practice  quizzes  and  online  tutorials)  can  be  managed  by  free  online  resources. 

It  is  not  the  intention  of  this  paper  to  suggest  that  instructors  or  academic  institutions  replace  their 
commercial  LMS  with  free  online  resources,  but  free  online  resources  should  be  reserved  as  a viable  and  beneficial 
option  for  instructors  and  tutors  that  do  not  have  access  to  their  own  LMS.  The  above  data  can  also  influence 
administrators  and  instructors  when  choosing  to  implement  and  LMS  and  which  LMS  features  would  be  most 
appropriate  for  their  school  (Black  et  al.,  2007). 

The  data  presents  a striking  question:  Why  do  students  report  great  value  in  resources  that  they  don’t 
utilize,  and  utilize  resources  that  they  say  have  limited  subjective  value?  It  may  be  the  case  that  the  wording  of  the 
survey  questions  skewed  the  data  by  asking  whether  specific  functions  were  “helping  students  achieve  their 
academic  goals”.  Perhaps  surveyed  students  do  not  interpret  specific  resources  like  uploading  of  files  valuable  in 
achieving  mastery  subject  matter,  but  find  that  they  save  time  or  are  helpful  in  another  manner.  Future  surveys 
should  make  sure  to  distinguish  how  these  resources  are  perceived  helpful.  It  is  clear,  though.that  This  paper 
supports  the  work  of  Pilarski  who  suggests  that  having  online  lectures  available  reduces  student  anxiety. 

Considering  this,  it  may  also  be  the  case  that  the  resources  in  question  are  considered  vital  emergency  resources  but 
not  useful  day-  to-  day.  An  appropriate  analogy  is  that  of  the  spare  tire  in  a car.  It  is  not  needed  often,  but  when 
needed  it  is  invaluable.  This  would  certainly  be  the  case  for  students  that  missed  a vital  end  of  term  lecture. 

Whatever  the  reason  for  the  low  utilization  rates  but  high  perceived  subjective  values  reported,  instructors  should  be 
cautioned  to  look  deeper  than  simple  utilization  rates  when  deciding  which  resources  to  put  their  energy  into 
developing. 

It  is  evident  that  more  rigorous  and  expanded  research  is  needed  to  address  the  question  of  whether  the 
techniques  utilized  in  this  course  actually  increase  student  GPA  and  retention  and  ultimately  impact  the  number  of 
students  applying  for  admission  to  professional  healthcare  programs.  It  is  our  position  that  additional  research  will 
show  that  the  use  of  an  LMS,  social  networking,  and  mentoring  are  synergistic  in  their  effects.  It  is  often  true  that 
the  more  resources  that  are  made  available  to  the  student,  the  better  their  achievement  (Martinez,  2000).  However, 
the  ease  with  which  instructors  can  use  social  networks  and  file  sharing  abilities  makes  these  resources  of  particular 
value. 

Lastly,  while  the  literature  shows  that  tutoring  increases  achievement  (Gibbons  and  Dixon,  2001),  and  peer 
groups  influence  academic  success  (Sallee  and  Tierney,  2009),  further  research  is  needed  to  validate  the  premise  that 
the  use  of  an  LMS  increases  the  chance  that  students  will  stay  in  contact  with  their  tutors  and  fellow  students  and 
thereby  increase  their  retention  and  GPA. 
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Conclusions 

We  close  with  seven  recommendations  for  practitioners: 

1 . Students  should  be  encouraged  to  use  social  networking  features  of  the  existing  LMS  to  form  study  groups 
since  the  LMS  can  support  many  varied  user  group  functions  and  students  have  shown  that  they  will  form 
support  groups  with  or  without  an  LMS. 

2.  Instructors  should  participate  in.  and  make  use  of,  the  social  networking  aspects  of  their  academic 
institution’s  LMS  (wikis,  blogs,  etc.)  Further,  academic  institutions  should  promote  and  train  instructors  to 
become  more  comfortable  with  the  social  networking  aspects  of  the  available  online  networks. 

3.  Considerations  for  implementing  or  adopting  an  LMS  should  include  an  evaluation  of  its  social  networking 
functions. 

4.  The  most  commonly  used  features  of  a typical  and  traditional  LMS  (e.g.  file  sharing)  should  be  included  in 
popular  online  social  networks. 

5.  Instructors  should  think  critically  when  deciding  what  resources  to  make  available  to  students.  Just  because 
students  don't  regularly  use  a resource,  doesn't  mean  they  don't  need  it  or  want  it  available. 

6.  Tutors,  mentors,  and  college  tutoring  centers  should  increase  their  effectiveness  and  subjective  value  to 
students  by  using  online  social  networks  to  make  themselves  more  available  to  students.  Additionally,  the 
use  of  social  networking  tools,  while  increasing  availability,  can  also  allow  the  tutor  or  instructor  to  set 
healthy  boundaries  because  the  educator  can  choose  when  to  check  their  online  status  and  those  of  the 
students. 

7.  Future  longitudinal  research  is  needed  to  show  how  GPA  and  retention  are  affected  by  the  use  of  LMSs  and 
social  networks. 


References 

American  Association  of  Colleges  of  Nursing  (AACN).  (2008).  Nursing  Shortage  Fact  Sheet.  Available  online. 
Accessed  February  10,  2009.  http://www.aacn.nche.edu/Media/FactSheets/NursingShortage.htm 

Beisser,  Sally  R.  "Technology  Mentorships  in  Higher  Education:  An  Optimal  Match  for  Expanding  Educational 
Computing  Skills."  (2000):  ERIC.  EBSCO.  Web.  22  Oct.  2009. 

Blackman,  Ian,  Margaret  Hall,  and  1 Gusti  Ngurah.  Darmawan  "Undergraduate  Nurse  Variables  that  Predict 

Academic  Achievement  and  Clinical  Competence  in  Nursing."  International  Education  Journal  8.2  (2007): 
222-236.  ERIC.  EBSCO.  Web.  22  Oct.  2009. 

Biemiller,  Lawrence  "Bursting  at  the  Seams."  Chronicle  of  Higher  Education  55.29  (2009):  Bll.  ERIC.  EBSCO. 
Web.  22  Oct.  2009. 

Black,  E.  W.,  Beck,  D.,  Dawson,  K.,  Jinks,  S.,  & DiPietro,  M.  (2007,  Mar/Apr).  The  other  side  of  the  LMS: 
Considering  implementation  and  use  in  the  adoption  of  an  LMS  in  online  and  blended  learning 
environments.  TechTrends,  51(2),  pp.  35-39,  53. 

Branzburg,  Jeffrey  "How  to  Use  the  Moodle  Course  Management  System."  Technology  & Learning  26.1  (2005):  40. 
ERIC.  EBSCO.  Web.  22  Oct.  2009. 

Brown,  J.  S.,  Collins,  A.,  and  Duguid,  P.  1989.  Situated  cognition  and  the  culture  of  learning.  Educational 
Researcher,  41:32-42. 

Buerhaus,  P.,  Potter,  V.,  Staiger,  D.,  Auerbach,  D.  (2009).  The  Future  of  the  Nursing  Workforce  in  the  United 
States:  Data,  Trends  and  Implications.  Sudbury,  MA:  Jones  and  Bartlett  Publishers. 

Ciani,  Keith  D.,  et  al.  "Collaborative  Learning  and  Positive  Experiences:  Does  Letting  Students  Choose  Their  Own 
Groups  Matter?."  Educational  Psychology  28.6  (2008):  627-641.  ERIC.  EBSCO.  Web.  22  Oct.  2009. 

Clark,  Y.  Mentoring  Students  Online.  In  Penn  State  Education  Technology  Sendees.  Retrieved  September  20,  2009, 
from  http://tlt.its.psu.edu/suggestions/mentor/. 

Fox,  Seamus,  and  Kay  MacKeogh  "Can  eLearning  Promote  Higher-Order  Learning  without  Tutor  Overload?."  Open 
Learning  18.2  (2003):  121-134.  ERIC.  EBSCO.  Web.  22  Oct.  2009. 

Gribbons,  Barry  C,  P.  Scott  Dixon,  and  Santa  Clarita,  CA.  Office  of  Institutional  Development.  College  of  the 
Canyons  "College  of  the  Canyons  Tutoring/Learning/Computer  Center  Retention  and  Success,  Spring 
2001.  Report."  (2001):  ERIC.  EBSCO.  Web.  22  Oct.  2009. 


401 


Hung,  David  "Situated  Cognition  and  Problem-Based  Learning:  Implications  for  Learning  and  Instruction  with 
Technology."  Journal  of  Interactive  Learning  Research  13.4  (2002):  393-414.  ERIC.  EBSCO.  Web.  22 
Oct.  2009. 

Imel,  Susan,  and  Career,  and  Vocational  Education,  Columbus,  OH.  ERIC  Clearinghouse  on  Adult  Adult  Learning 
in  Cohort  Groups.  Practice  Application  Brief  No.  24.  2002.  ERIC.  EBSCO.  Web.  22  Oct.  2009. 

Lave,  L,  Wenger,  E.  Situated  Learning:  Legitimate  Peripheral  Participation.  New  York,  Cambridge  University 
Press.  1990. 

Martinez,  Paul,  and  London  (England).  Further  Education  Development  Agency  Reusing  Achievement:  A Guide  to 
Successful  Strategies.  2000.  ERIC.  EBSCO.  Web.  22  Oct.  2009. 

O'Donnell,  Victoria  L.,  and  Jane  Tobbell  "The  Transition  of  Adult  Students  to  Higher  Education:  Legitimate 

Peripheral  Participation  in  a Community  of  Practice?."  Adult  Education  Quarterly:  A Journal  of  Research 
and  Theory  57.4  (2007):  312-328.  ERIC.  EBSCO.  Web.  22  Oct.  2009. 

Okezie,  Chukwunyere  E.  "Of  Deconstruction  and  Reconstruction:  Confronting  the  Challenges  of  the  Adult 

Learner."  Negro  Educational  Review  54.3/4  (2003):  1 15-119.  Academic  Search  Premier.  EBSCO.  Web.  19 
Oct.  2009. 

Pilarski,  Piotr  P.,  et  al.  "From  music  to  macromolecules:  Using  rich  media/podcast  lecture  recordings  to  enhance  the 
preclinical  educational  experience."  Medical  Teacher  30.6  (2008):  630-632.  Academic  Search  Premier. 
EBSCO.  Web.  19  Oct.  2009. 

Rogers,  James  W.,  and  James  R.  Cox  "Evaluating  a New  Online  Course  in  the  Epidemiology  of  Infectious  Diseases 
by  Studying  Student  Learning  Styles."  Journal  of  College  Science  Teaching  37.3  (2008):  29-33.  ERIC. 
EBSCO.  Web.  22  Oct.  2009. Sallee,  Margaret  W.,  and  William  G.  Tierney  "The  Influence  of  Peer  Groups 
on  Academic  Success."  College  & University  82.2  (2007):  7-14.  ERIC.  EBSCO.  Web.  22  Oct.  2009. 


402 


Flexible  Online  Learning  and  Game-based  Learning  Environments: 
Do  They  Make  a Happy  Marriage? 

Kajal  Shah 

N057  Lagomarcino  Hall 

Center  for  Technology  in  Learning  and  Teaching 
Department  of  Curriculum  and  Instruction 
Iowa  State  University 
Ames,  Iowa  50011,  USA 

Ana-Paula  Correia 
N165  B Lagomarcino  Hall 
Center  for  Technology  in  Learning  and  Teaching 
Department  of  Curriculum  and  Instruction 
Iowa  State  University 
Ames,  Iowa  50011,  USA 

Turkan  Karakus 

Department  of  Computer  Education  and  Instructional  Technology 
Middle  East  Technical  University 
06531  Cankaya,  Ankara,  TURKEY 

Keywords:  Distance  Education,  Gaming/Simulation 


Abstract 

This  literature  review  and  reflection  paper  explores  the  potential  of  online  game -based  environments  to 
enhance  e-learning  experiences  and  suggests  a possible  marriage  between  them  where  gaming  environments  could 
serve  as  a platform  for  online  learning.  This  is  placed  within  the  context  of  the  new  directions  that  learning  has  to 
take  in  order  to  engage  the  net  generation  (Gen-N).  It  outlines  the  principles  of  distance  learning  and  the  learning 
principles  that  underlie  game  design  to  examine  their  similarities  and  see  if  they  could  make  a happy  marriage.  In 
conclusion,  the  paper  cites  initiatives  already  taken  in  this  direction  of  using  game  based  virtual  environments  in 
distance  education.  The  paper  is  based  mostly  on  literature  survey  of  articles  and  books  on  gaming  and  education 
and  on  authors’  experience  in  gaming  and  online  learning. 

Introduction 

With  the  advent  and  growing  popularity  of  virtual  3-D  worlds  and  highly  graphical,  realistic  online  gaming 
environments,  the  opportunities  for  collaboration  in  online  environments  have  taken  a new  direction.  . The  net 
generation  (Gen-N),  used  to  instant  ‘one  touch'  information  access  and  communication,  are  learning  informally  in 
a variety  of  ways  by  creating  and  collaborating  and  their  brains  and  thinking  processes  are  also  likely  to  have  been 
shaped  by  very  visual,  rapid-moving,  hyper-texted  environments  (Annetta,  Murray,  Laird,  Bohr  & Park,2006).  Even 
as  schools  and  teachers  struggle  to  keep  up  with  the  changing  technology,  they  are  impeded  by  their  adherence  to 
traditional  pedagogical  beliefs  and  practices  (formal,  instructor-controlled,  one-to-many,  linear,  passive  learning  of 
facts  organized  in  discrete  chunks  with  no  connection  to  self).  Despite  several  initiatives  to  incorporate  active 
learning  through  technology,  change  in  schools  is  slow  to  come  as  teachers  fail  to  relate  with  computers  and  ICT 
(Internet  and  Communications  Technology)  in  the  way  their  students  do.  They  are  uneasy  with  the  change  in 
teacher  roles  that  technology-mediated  learning  necessitates  and  fear  a loss  of  control. 

In  contrast  to  classroom  instruction,  distance  education,  being  primarily  mediated  through  electronic  forms 
of  communication  and  access,  is  uniquely  equipped  to  better  integrate  new  technologies  into  its  design  and  shape  a 
different  learning  experience  for  students.  This  is  also  made  possible  by  the  underlying  philosophy  that  guides  the 
principles  of  good  practice  in  flexible  distance  education  which  emphasizes  multiple  modes  of  communication  and 
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access,  responsiveness  to  student  needs  and  learning  preferences,  multiple  means  of  content  delivery,  clear  learning 
objectives,  learning  in  social  contexts,  social  presence,  student  collaboration  and  building  of  learning  communities, 
prompt  feedback,  reflexivity  in  learning,  self-regulated  participation  and  flexibility,  which  give  the  learner  more 
control  over  the  pace,  place  and  time  of  learning. 

However,  the  promise  of  distance  learning  has  not  been  realized  and  with  a few  exceptions,  e-learning 
environments  continue  to  be  seeped  with  traditional  teaching  strategies  and  have  come  to  replicate  the  social 
organization  of  traditional  schooling.  Dutton  (2004)  cites  the  case  of  a university  where  the  virtual  learning 
environment  was  used  as  nothing  more  than  a replacement  for  the  copy  machine  and  projector,  without  modifying 
underlying  approaches  to  teaching  and  learning  for  an  online  environment. 

The  goal  of  this  study  is  to  provide  some  insights  related  to  how  game-based  learning  environments  can 
provide  a platform  for  flexible  online  learning.  It  provides  good  recommendations,  especially,  for  online  education 
practitioners  who  seek  new  solutions  to  make  online  learning  more  appealing  and  engaging.  Also  game-based 
environment  developers  could  benefit  from  this  reflection  on  overcoming  some  challenges  when  designing  games 
for  online  learning.  This  study  was  the  result  of  a literature  review,  personal  experiences  on  using  games  and 
developing  online  education,  and  analysis  of  game -based  and  online  learning  environments. 

Flexible  Online  Learning:  A Promise  Unrealized 

E-leaming  has  become  extremely  popular  with  full  time  working  professionals  and  people  who  are  hard 
pressed  for  time  to  attend  a university,  and  with  corporate  organizations  who  see  in  this  a convenient  means  to 
deliver  training  to  its  employees.  The  demand  for  this  type  of  instruction  is  high  and  because  it  can  be  cheap  (an 
entire  course  can  be  delivered  via  free  email),  everyone  wants  to  get  in  the  fray  and  capitalize  on  it,  with  little  regard 
to  flexible  online  learning  principles.  In  schooling  also,  distance  learning  is  sometimes  no  more  than  traditional 
lectures  delivered  over  the  webcam. 

This  has  prompted  Aldrich  (2004)  to  comment  that  “So  far,  e-learning  has  made  about  the  same 
contributions  to  learning  as  fast  food  has  made  to  food”.  He  points  out  that  both  industries  are  driven  by  low-cost, 
high  profit  goals.  Their  products  allow  for  minimal  deviation  from  standards.  Development  and  consumption  must 
be  simple  and  rapid  - get  in,  get  something  from  the  standard  menu,  get  out,  in  a world  of  'drive-through  content'.  In 
one  of  the  most  scathing  criticisms  of  e-learning.  Squire  (2005,  p.5)  says  that  ‘...as  it  is  currently  constituted,  e- 
learning  is  an  evolution  in  education,  not  a revolution’ . 

“The  basic  organizing  metaphors  of  traditional  education  - knowledge  as  discreet  and  abstract  facts, 
learning  as  the  “acquisition”  of  content,  and  therefore  instruction  as  the  organization,  dissemination,  and 
management  of  that  content  - have  gone  unchanged.  The  promise  of  e-Learning  - that  it  would  provide 
customized,  accessible  learning  experiences  - has  given  way  to  more  mundane  pursuits,  such  as  free  online 
content.” 

Further,  he  states  that: 

“Publishing  content  online  is  not  synonymous  with  making  learning  accessible,  or  actually  ensuring 
learning.  E-Learning  educators  have  focused  too  much  on  the  “e”  - making  content  electronic  (or  more 
accurately,  digital)  - and  not  enough  on  the  learning  - creating  technology  enhanced  experiences  designed 
to  change  future  understanding  and  performance.  E-learning  has  become  all  about  replicating  traditional 
education  in  an  electronic  format.  In  short,  we  have  become  experts  at  technology-mediated  chalk  and 
talk.” 


Digital  Games  in  Flexible  Online  Learning:  A New  Paradigm? 

What  has  been  more  difficult  to  achieve  is  the  effective  design  of  instruction  in  order  to  provide  the  kind  of 
social  and  material  experiences  necessary  to  make  sense  of  that  content,  and  to  make  it  meaningful  and  useful  for 
future  action.  Video  games  designers  have  been  very  successful  in  this  aspect.  With  modest  beginnings  in  two 
dimensional  arcade -type  games  with  rougher  graphics  and  sound,  requiring  no  more  skill  than  just  hand-eye  co- 
ordination (called  twitch  games),  game  design  has  progressed  to  include  simulations  and  three  dimensional  multi- 
user role-playing  games  with  strong  narratives  and  character  development,  which  immerse  users  in  rich  interactive 
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digital  microworlds  (Lim,  2008;  Squire,  2003).  According  to  Foreman  (2003)  this  new  genre  of  games  provide  such 
engaging,  rapid  paced,  graphically  rich  and  dynamic  experiences  that  they  make  any  sort  of  conventional 
schoolwork  mediated  through  lectures  or  text,  seem  boring  by  comparison. 

Game-based  learning  environments  have  being  used  to  improve  students’  critical  thinking,  inquiry  skills 
and  communication  skills,  making  them  candidates  of  effective  e-learning  platforms  (BurgosTattersall,  and  Koper, 
2007).  The  motivation,  active  participation,  sense  of  community  and  critical  thinking  approach  possible  in  game 
based  learning  environments  could  be  integrated  with  content  delivery  in  e-learning.  . For  instances.  Gee’s  (2003) 
work  on  gaming  advocates  learning  as  a state  of  playing  where  learners  work  with  other,  engage  in  activities,  solve 
problems,  share  knowledge,  and  evaluate  other  and  themselves  within  the  learning  environment.  Dede  (2003) 
predicts  that  game-based  environments  will  be  one  of  the  common  interfaces  that  people  will  interact  with  in  the 
upcoming  decades.  Both  scholars  present  futuristic  scenarios  in  which  learning  takes  place  in  game-based 
environments,  and  the  separation  between  school  and  home,  learning  and  entertainment  vanish.  In  this  line  of 
reasoning,  Dede  (2005 ) defines  the  term  “neomillenial”  learning  style.  Fie  argues  that  although  game-based  learning 
environments  may  not  provide  real  life  learning,  they  can  support  neomillinial’s  learning  needs  by  providing 
opportunities  for  intense  interaction  and  fostering  higher  order  inquiry  skills.  To  make  this  integration  seamless, 
some  issues  should  be  clarified  and  some  challenges  should  be  considered  before  proposing  a marriage  between 
these  two  approaches. 

The  Basis  for  a Marriage  between  Flexible  Online  Learning  and  Game-based  Learning  Environments 

Game  analysis  reveals  that  game  designers  craft  fun,  engaging  and  compelling  experiences  by  balancing 
several  components  of  the  game,  namely,  characters  (fantasy),  game  rewards  (motivation),  obstacles  (challenge), 
narrative  (curiosity),  competition  (with  self  and  other  humans)  and  opportunities  for  collaboration  with  other  players 
(social  recognition)  (Annetta  et.al,  2006;  Squire,  2003)  . Protagonists  for  incorporating  games  into  education  find 
that  games  embody  well  established  principles  and  models  of  learning  which  are  in  line  with  constructivist 
principles. 

One  of  the  principles  they  identify  is  “situated  cognition’’  (Van  Eck,  2006;  Squire,  2005),  which  emphasizes  that  for 
effective  learning  to  happen,  it  has  to  be  rooted  in  direct  experience  and  concrete  contexts.  For  example,  games  give 
players  clear  goals  that  must  be  achieved  and  tasks  that  lead  up  to  it.  Thus  learning  tasks  are  placed  within  the 
meaningful  context  of  the  game.  Moreover,  whatever  learning  that  happens  is  tied  to  the  achievement  of  these  goals 
and  has  to  be  demonstrated  within  the  game  environment.  Thus,  learning  is  made  relevant  with  direct  practical 
application.  According  to  Aldrich  (2004),  games  follow  the  advice  that  Aristotle  dispensed  a millennia  ago:  the  only 
way  to  learn  is  by  doing.  Principles  behind  immersive  gaming  experiences  clearly  show  that  they  are  in  complete 
alignment  with  the  principles  of  constructivist  flexible  distance  learning  environments.  In  the  authors’  view,  it  would 
be  easier  to  integrate  game -based  learning  into  something  which  naturally  tries  to  create  an  immersive,  collaborative 
environment  than  a conventional  classroom,  where  the  ecology  of  teaching  is  contrary  to  the  informal  learning 
through  games.  For  instance,  the  role  of  the  instructor  in  flexible  online  teaching  is  understood  to  be  more  of  a 
facilitator,  nudging  students  to  think  in  different  ways  and  providing  for  opportunities  for  collaboration  and  in  a 
game  based  environment,  this  role  would  remain  unchanged.  Furthermore,  gaming  occurs  in  rich  socio-cultural 
contexts,  which  is  apparent  in  the  several  forums  and  blogs  that  have  sprung  up  around  multi-user  games,  where 
gamers  share  their  experiences,  strategies  that  worked,  and  provide  suggestions  to  each  other  for  improvement,  in 
essence,  forming  communities  of  practice,  which  is  also  one  of  the  main  goals  in  flexible  online  learning. 

Role  playing  simulation  and  strategy  games  such  as  Civilization,  Sims,  Age  of  Empires,  Roller  Coaster 
Tycoon,  Dungeons  and  Dragons  (all  commercial,  off  the  shelf  games  or  COTS ) place  players  in  a problem-solving 
role  that  requires  them  to  make  frequent  important  decisions  which  decides  the  course  of  the  game  and  its  outcome. 
For  instance,  in  Dungeons  and  Dragons  the  player  is  sent  on  a quest  to  find  a cure  for  a curse  which  plagues  this 
fantasy  land.  The  players  have  to  gather  information  from  characters  by  ‘talking’  to  them,  and  some  characters 
might  try  to  mislead  them,  so  they  have  to  evaluate  all  the  information  and  take  a decision.  Players  have  to  acquire 
objects  (spells,  swords)  and  skills  and  gain  training  in  them  and  there  is  action  in  the  form  of  duels  and  fights  with 
evil  characters.  Thus,  not  only  is  the  game  narrative  varied  and  unpredictable,  but  also  encourage  inquiry, 
information  gathering  and  reflection  (Lim,  2008)  and  in  multi-user  environments,  encourages  collaboration  and 
information  sharing,  which  is  often  done  through  text  or  voice  chat  capabilities  inbuilt  in  the  gaming  environment. 
According  to  Van  Eck  (2006),  this  whole  process  engages  players  in  a ‘Piagetian’  cycle  of  ‘assimilation’  (fitting  new 
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information  into  existing  slots  and  categories)  and  ‘accommodation'  or  (modifying  understanding  to  make  sense  of 
information  that  does  not  fit  into  existing  slots)  requiring  a constant  cycle  of  hypothesis  formulation,  testing  and 
revision. 

Attempts  by  educational  designers  to  create  games  have  met  with  limited  success,  because  they  ‘smell'  too 
much  like  school  (the  educational  emphasis  being  more  apparent  than  the  experiential  part)  and  students  soon 
become  disenchanted.  This  has  led  to  the  ‘serious  gaming'  movement,  where  educationists,  military,  corporations, 
medical  fields  and  game  designers  collaborate  to  creating  games  to  engage  players  in  critical  content  with 
experiential,  task-based  activities  situated  in  multiplayer  online  3-D  virtual  environments  rich  in  graphical 
representations,  objects,  tools,  avatars  (virtual  representations  of  each  student)  and  required  instructional  content 
material. 

River  City,  a serious  gaming  project  initiated  by  Harvard  in  the  Active  Worlds  Multi-user  Virtual  Environment 
incorporates  many  of  the  principles  of  role-playing  strategy  games  mentioned  above  such  as  problem  solving, 
information  seeking  (clues),  collaboration,  information  sharing,  assimilation,  accommodation,  skill  development  and 
situated  cognition  within  a rich  story-line.  The  narrative  is  based  on  authentic  historical  conditions  where  students, 
working  together  in  small  teams,  have  to  discover  clues,  analyze  information,  develop  hypotheses  and  test  them  to 
propose  a solution  to  the  causes  of  illness  in  a 19th  century  town.  Thus,  in  a game -like  environment,  students  leant 
to  carry  out  an  authentic  scientific  inquiry  (Ketelhut,  Dede,  Clarke  & Nelson,  2006)  which  also  helps  to  enhance 
communication  and  problem  solving  skills  (Dieterle  & Clarke,  2008).  Other  game-based  learning  environments  - 
Quest  Atlantis  (IndianaUni versify).  Environmental  Detectives  (MIT),  Hazmat:Hotzone  (Carnegie  Mellon), 
Revolution  (MIT)  are  also  good  examples  to  serve  constructivist  learning  environments. 

However  simply  using  a simulated  environment  does  not  ensure  that  learners  will  generate  the  kinds  of 
understanding  that  educators  might  desire.  The  learning  has  to  be  “made  visible”  to  them  by  providing  debriefing 
and  opportunities  to  reflect  on  their  activities.  Instructors  play  an  important  role  in  this  process  fostering 
collaboration,  promoting  reflection  and  coordinating  extension  activities.  Thus,  the  potential  for  structuring 
educational  activities  in  a manner  that  generates  such  a state  of  engagement  along  with  appropriate  scaffolding 
provides  the  required  direction  for  using  game -based  learning  in  schools. 

One  of  the  major  challenges  in  this  marriage  is  to  overcome  the  time  and  resources  investment  required  for 
game-based  learning  environments  (Prensky,  2001).  Another  challenge  in  integrating  games  into  flexible  online 
learning  is  putting  instructional  designers,  course  instructors  and  game  developers  into  the  same  team  (Kirkley  & 
Kirkley,  2004).  Available  instructional  design  methodologies  do  not  meet  the  needs  of  design  and  development  of 
complex,  virtual,  game-based  environments.  As  Kirkley  & Kirkley  (2004)  explained  a story  board  might  mean 
different  things  for  instructional  designers  and  game  developers.  One  way  to  approach  this  challenge  is  by 
promoting  the  use  of  games  in  teacher  education  programs  (Shah,  Correia  & Karakus,  2009).  Course  instructors  may 
design  instructional  activities  so  that  pre-service  teachers  can  better  articulate  the  use  of  gaming  in  their  own 
subjects  and  even  create  initial  prototypes  designed  for  their  students  in  a virtual  schooling  setting. 

Another  issue  seems  to  be  the  students/players’  evaluation  in  game-based  learning  environments  when 
integrated  in  online  learning.  One  way  to  overcome  this  issue  is  to  embed  evaluation  systems  into  the  environments 
that  check  students/players’  cognitive  and  affective  changes  while  they  engage  in  the  learning  activities.  They  can  be 
rewarded  and/or  given  alternative  activities  to  compensate  for  any  mistakes.  This  might  be  controlled  by  agents  and 
avatars.  At  the  same  time  online  logs  and  records  can  provide  additional  evaluative  data. 

Conclusions 

The  game-based  learning  environments  described  above  represent  a new  paradigm  in  distance  learning  and 
themselves  form  the  platform  of  delivery,  making  the  important  distinction  between  “using”  games  (implying  that 
games  be  present  in  the  course  of  instruction)  and  “integrating”  them  (involving  careful  analysis  and  alignment  with 
instructional  strategies  and  learning  outcomes  and  intelligent  implementation)  into  a distance  learning  environment. 
Tools  such  as  virtual  notebook,  text  based  chat  allow  students  to  interact  and  keep  /share  notes  of  their  observations 
and  data  collection,  also  providing  a place  for  reflection.  Although  not  perfect,  these  examples  represent  important 
experiments  in  gaming  and  online  learning  and  form  the  foundations  which  will  redefine  distance  learning  along 
new,  exciting  lines. 

Flexible  online  learning  and  game -based  learning  environments  are  both  based  on  similar  constructivist 
learning  models,  and  are  not  antithetical  to  the  purpose  of  education.  Online  learning  can  profit  substantially  by 
incorporating  the  motivational  and  immersive  principles  used  by  game  designers.  Online  learning  environments 
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integrated  with  game-based  learning  open  up  the  possibilities  of  using  games  as  spaces  for  experiential  learning,  as 
contexts  for  discussion,  as  tools  to  think  with  and  as  an  engaging  arena  for  meaningful,  collaborative  learning. 

As  game-based  learning  environments  become  more  pervasive  in  children’s  lives,  educators  will  also  have 
to  be  prepared  to  consider  the  effects  of  these  experiences  on  children’s  identities  and  development,  along  with 
organizational,  political,  economic  and  cultural  obstacles  (Dede,  2002)  to  using  game-based  learning  environments 
as  flexible  learning  tools. 


References 

Aldrich,  C.  (2004).  Simulations  and  the  future  of  learning:  An  innovative  (and  perhaps  revolutionary ) approach  to 
e-learning.  San  Francisco:  John  Wiley. 

Annetta,  L.A.,  Murray,  M.R.,  Laird,  S.G.,  Bohr,  S.C.  & Park,  J.C.  (2006).  Serious  Games:  Incorporating  video 
games  in  the  classroom.  Educause  Quarterly,  3,  16-22. 

Burgos,  D.,  Tattersall,  C.  and  Koper,  R.  (2007).  Re-purposing  existing  generic  games  and  simulations  for  e- 
learning,  Computers  in  Human  Behaviour,  23(6),  2656-2667 

Dede,  C.  (2002).  Vignettes  about  the  future  of  learning  technologies.  In  R.  Bajcsy  (Ed.),  2020  VISIONS: 

Transforming  Education  and  Training  through  Advanced  Technologies  (pp.  18-25).  Washington,  DC:  U.S. 
Department  of  Commerce. 

Dede,  C.  (2003).  Multi-user  virtual  environments.  Educause  Review,  38(3),  60-61. 

Dede,  C.  (2005).  Planning  for  neomillennial  learning  styles:  implications  for  investments  in  faculty  and  technology. 
In  D.  Oblinger  & J.  Oblinger  (Eds.),  Educating  the  Net  Generation  (pp.  15.1-15.22).  Boulder,  CO: 
EDUCAUSE.  Retrieved  October  1,  2009,  from  http://www.educause.edu/educatingthenetgen 

Dieterle,  E.,  & Clarke,  J.  (2008).  Multi-user  virtual  environments  for  teaching  and  learning.  In  M.  Pagani  (Ed.), 
Encyclopedia  of  Multimedia  Technology  and  Networking,  2nd  ed.  (pp. 1033-1041).  Hershey,  PA:  Idea 
Group. 

Dutton,  W.H.  (2004).  Social  transformation  in  an  information  society:  rethinking  access  to  you  and  the  world.  Paris: 
United  Nations  Educational,  Scientific  and  Cultural  Organization  7.  Retrieved  February  1,  2006,  from 
http://portal.unesco.org/ci/en/files/12848/11065568745corpus-l-144.pdf/corpus-l-144.pdf 

Gee,  J.P.  (2003).  What  video  games  have  to  teach  us  about  learning  and  literacy.  Computers  in  Entertainment,  1(  1), 
1-4. 

Foreman,  J.  (2003).  Next  generation  educational  technology  versus  the  lecture.  EDUCAUSE  Review,  retrieved 
September  18,  2009,  from  http://www.educause.edu/ir/library/pdf/erm0340.pdf 

Ketelhut,  D.  J.,  Dede,  C.,  Clarke,  J.,  & Nelson,  B.  (2006).  A multi-user  virtual  environment  for  building  higher 
order  inquiry  skills  in  science.  Paper  presented  at  the  American  Educational  Research  Association  2006 
Annual  Meeting,  San  Francisco,  California,  April  7-11,  2006. 

Kirkley,  S.E.  & Kirkley,  J.  (2004).  Creating  Next  Generation  Blended  Learning  Environments  Using  Mixed  Reality, 
Video  Games  and  Simulations.  TechTrends,  49(3),  42-53 

Lim,  C.P.  (2008).  Spirit  of  the  game:  empowering  students  as  designers  in  schools?  British  Journal  of  Educational 
Technology,  39  (6),  996-1003. 

Prensky,  M.  (2001).  Digital  Game  Based  Learning.  New  York:  McGraw-Hill. 

Shah,  K.,  Correia,  A.P.  & Karakus,  T.  (2009).  Alternative  Technology  Exploratorium:  Efforts  to  bring  main  stream 
gaming  into  teacher  education.  Paper  presented  at  the  5,h  Games+Learning+Society  Conference,  Madison, 
WI,  June  10-12,2009. 

Squire,  K.  (2003).  Video  games  in  education.  International  journal  of  intelligent  simulations  and  gaming,  2 (1),  49- 
62 

Squire,  K.  (2005).  Game-Based  Learning:  Present  and  Future  State  of  the  Field.  An  x-Learn  Perspective  Paper. 
Retrieved  May  8,  2007,  from  http://cohesion.rice.edu/Conferences/Hewlett/emplibrary/Game- 
Based  Learning.pdf 

Van  Eck,  R.  (2006).  Digital  game -based  Learning:  it’s  not  just  the  digital  natives  who  are  restless.  Educause  Review, 
41(2),  16-30. 


407 


Using  Activity  Theory  to  Guide  E-Learning  Initiatives 

Neal  Shambaugh,  neal.shambaugh@mail.wvu.edu 

Abstract 

Activity  theory’s  visual  and  conceptual  representation  of  activity-in-context  provided  an  analysis  and  synthesis  tool 
to  help  department  faculty  begin  to  develop  an  online  instructional  design  and  technology  (IDT)  master’s  program. 
Analyzing  different  activity  systems  for  students,  faculty,  and  administrators  revealed  that  E-Learning  goals 
overlapped,  but  differences  existed  in  terms  of  rules-norms,  community,  and  roles.  Analysis  results  were  then 
organized  by  existing  faculty,  student,  and  administrator  concerns  with  potential  boundary-crossing  actions 
(Engestrom,  2002).  This  paper  first  discusses  the  need  for  simultaneous  E-Learning  curriculum  and  administrative 
decision-making,  then  reports  progress  across  four  stages  of  developing  the  online  program.  Three  implications  of 
activity  theory  for  E-Learning  are  discussed.  Guidelines  for  using  activity  theory  in  program  development  are 
described. 


Background 

Administrators  push  for  E-Learning  initiatives,  which  increase  student  enrollments  but  require  an 
increasing  amount  of  faculty  and  administrative  time.  Faculty  are  increasingly  called  on  to  revise  or  develop  new 
programs  but  also  to  develop  plans  to  market,  recruit,  and  retain  students,  tasks  that  are  new  and  foreign  to  them. 
Faculty  groups  work  with  little  organizational  experience  to  implement  E-Learning  initiatives.  Traditional  models  of 
curriculum  development  take  too  long  and  academic  organizations  are  not  geared  to  address  immediate  needs  and 
understand  the  complex  contexts  surrounding  fast-changing  market  conditions  that  academia  finds  itself  in. 

Activity  theory  has  been  used  to  analyze  educational  settings  ranging  from  computer-based  training  to 
better  understand  the  workplace  in  which  the  training  was  used  (Pang  & Hung,  2001),  as  well  to  acknowledge 
teachers’  beliefs  about  teaching  and  the  power  issues  between  teachers  and  administrators  in  public  schools 
(Robertson,  2008).  Activity  theory  has  been  used  in  higher  education  strategic  E-Learning  initiatives  (Salomon, 
2005)  and  to  look  specifically  at  asynchronous  learning  networks  (Li  & Bratt,  2004). 

This  paper  documents  the  results  of  using  activity  theory  to  examine  the  issues  of  online  programs. 

Viewing  E-Learning  through  “activity”  acknowledges  the  different  constituencies  that  have  a stake  in  such 
programs,  as  well  as  the  context  of  curriculum  and  organizational  needs.  Nichols  (2008)  uses  the  three  activity 
systems  of  organizational  (management),  technological  (information  specialists),  and  pedagogical  (instructors)  to 
characterize  E-Learning.  In  this  paper,  the  three  activity  systems  are  viewed  as  faculty,  administrative  (department 
chairs),  and  students  are  analyzed.  Faculty  members  are  familiar  with  the  curriculum  issues  of  program  objectives, 
courses,  and  assessment.  Administrators,  meanwhile,  deal  with  student  recruitment,  course  staffing,  and  program 
coordination  and  evaluation.  E-Learning  initiatives  suggest  the  need  for  both  groups  to  address  curriculum  and 
organization  simultaneously.  From  an  activity  perspective,  two  activity  systems  are  in  play  with  overlapping  goals. 

A third  activity  system,  that  of  the  students,  represents  the  key  stakeholder. 

Conceptual  Framework  for  Using  Activity  Theory 

Activity  theory  is  a socio-cultural  perspective  on  understanding  the  interconnections  of  people, 
organizational  rules  and  culture,  and  mediating  tools,  all  directed  to  some  outcome  or  goal  (Bertelsen  & Bodker, 
2003;  Cole  & Engestrom,  1993).  Activity  theory  can  be  used  to  better  understand  the  goals  of  E-Learning  in  an 
academic  setting  and  in  a way  that  includes  all  of  the  major  constituents  and  the  influence  of  social  and  cultural 
norms,  values,  language,  and  tools  (Jonassen,  Tessmer,  & Hannum,  1999).  Representing  activity  as  a collective 
image  of  activity  (Engestrom,  1987,  p.  78)  includes  goals,  users,  and  tools,  and  the  rules  and  norms,  community 
practices,  and  division  of  labor. 

Specific  components  of  an  activity  system  model  for  E-Learning  initiatives  include  the  curricular  degree- 
granting program  connected  to  program  outcomes  (the  goal  of  the  activity  system);  users  including  students,  faculty 
members,  and  administrators;  the  division  of  labor  (i.e.,  who  does  what);  the  rules  and  norms  of  that  department;  the 
cultural  aspect  of  the  department,  or  unique  community  of  practice;  and  the  specific  E-Learning  technical  systems, 
known  in  the  theory  as  mediating  artifacts.  Specifically,  these  tools  include  the  online  learning  management  system 
and  the  administrative  system  (see  Figure  1). 
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Figure  1.  Activity  System  for  E-Learning. 


Using  Activity  Systems  for  E-Leaming  Analysis  and  Synthesis 

The  activity  system  visual  representation  provided  both  analysis  and  synthesis  opportunities.  E-Learning 
development  is  summarized  across  four  stages. 

Stage  1:  Analysis  Across  Three  Activity  Systems:  Faculty,  Students,  Administrators 

The  activity  theory  conceptual  structure  was  first  used  to  analyze  the  different  activity  systems  for  faculty, 
students,  and  administrators  (see  Figure  2).  Data  collection  occurred  during  a department  meeting  where  issues  were 
identified  and  afterwards  mapped  onto  the  activity  system  visual.  The  analysis  identified  an  overlapping  set  of  goals, 
which  suggested  that  E-Learning  was  a mutual  concern  and  that  changes  to  address  the  needs  of  one  group  would 
impact  one  or  more  of  the  other  two  groups  (see  Figure  3).  Differences  existed  in  terms  of  rules-norms,  community, 
and  roles.  A significant  overlap  of  concerns  existed  with  online  courses  and  advising,  meaning  that  any  implemented 
changes  will  impact  all  three  groups.  One  area  of  less  concern  for  faculty  and  students,  but  significant  for 
administrators  was  in  the  evaluating  and  improving  the  quality  of  the  overall  academic  programs,  including  E- 
Learning  components.  This  analysis  revealed  to  faculty  the  need  to  include  program  evaluation  in  the  design  of  all 
features  of  a program. 

Despite  overlapping  goals,  the  immediate  concerns  of  faculty,  students,  and  administrators  can  be  regarded 
as  “divided  terrain”  where  these  individuals  do  not  always  talk  or  work  with  each  other.  The  activity  system 
suggested  the  possibility  of  analyzing  a multitude  of  relations  within  the  triangular  structure  of  activity,  including 
shared  goals  and  differences  in  roles,  and  ways  of  working  together. 

The  overall  advantage  to  activity  theory  as  an  analysis  tool  is  that  the  model  raises  awareness  of  the  players 
to  contextual  and  historical  factors  that  comprise  human  activity.  The  different  players  can  then  discuss  the  potential 
impacts  of  this  context  on  reaching  the  goals.  Activity  theory  is  not  just  a front-end  analysis  tool,  as  needs 
assessment  is  in  instructional  design,  but  a “heuristic  aid”  (Gay  & Hembrooke,  2004)  for  the  players  to  continually 
evaluate  implementation  and  make  revisions,  but  also  to  keep  the  needs  and  concerns  of  the  interconnecting  players 
in  front  of  everyone. 
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Figure  2.  Three  E-Learning  Activity  Systems. 
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Figure  3.  Goal  Overlap. 


Stage  2:  Synthesis  of  Decisions  for  E-Learning 

The  use  of  activity  theory’s  conceptual  organization  has  helped  our  department  to  organize  around  a set  of 
discussion  points,  ones  that  address  the  concerns  and  realities  of  all  three  groups.  The  next  stage,  currently  ongoing 
through  the  academic  year  in  the  form  of  an  online  program  working  group,  is  to  document  issues  for  action  and  to 
make  recommendations  for  different  academic  programs.  Synthesis,  in  terms  of  decision-making,  is  now  being 
facilitated  by  discussing  how  the  use  of  E-Learning  tools  influences  or  mediates  the  goals  of  the  three  groups  and 
what  changes  in  mles-norms,  communities  of  practice,  and  roles  might  be  needed. 

At  this  second  stage,  online  courses  and  E-Learning  programs  require  what  Engestrom  (2002)  calls 
“boundary-crossing”  actions,  which  are  two-way  collaborative  interactions  requiring  both  renegotiation  and 
reorganization  decisions  (see  Figure  4).  Both  the  overlapping  and  differing  analysis  results  revealed  opportunities, 
which  have  forced  joint  responsibility  for  E-Learning  courses  and  programs  on  all  three  groups,  that  program 
development  in  E-Learning,  given  its  uniqueness  and  newness,  requires  an  ongoing,  iterative,  and  collaborative  set 
of  practices  (e.g.,  re-defining  student  contact  hour)  quite  different  from  traditional  academic  courses  and  programs. 
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Figure  4.  Boundary-Crossing  Opportunities. 

Examining  potential  boundary-crossing  issues  helped  to  develop  two  immediate  action  steps;  namely 
program  focus  and  marketing,  and  financial  support  for  further  development.  In  terms  of  Program  Focus  and 
Marketing,  faculty  identified  the  need  to  determine  student  focus  and  distinctive  program  features,  and  develop 
recruitment-to-graduation  support  system  (“paying  attention  to  online  student  needs”).  In  terms  of  Financial 
Support,  an  internal  grant  proposal  was  submitted  to  support  individual  faculty  development  of  online  courses  for 
IDT  Online  Master’s  Program.  Priorities  included  the  development  of  a common  “look  and  feel”  across  the  CMS 
product,  online  portfolio,  program  and  course  learning  outcomes,  and  appropriate  technology  tools  to  support  these 
outcomes. 

Stage  3:  Action  - Grant 

By  the  end  of  Spring  2009  a proposal  for  12  IDT  courses  was  submitted  to  the  institution’s  Extended 
Learning  Online  Program  Grants.  May  2009  the  department  was  awarded  $25,000  for  development  of  6 courses  to 
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roll  out  as  online  courses  beginning  summer  2010.  $4000  was  awarded  to  individual  course  faculty  members  after 
department  chair  sign-off  of  course  development  completed. 

Stage  4:  Current  Status  and  Action  Steps 

During  the  Fall  2009  semester  faculty  members  met  with  Extended  Learning  to  develop  a Memorandum  of 
Understanding  that  specified  activities  for  Extended  Learning  staff  (e.g.,  standardized  CMS  page  layout;  podcast 
creation)  and  individual  faculty  member  (outcomes,  assessment,  engagement  activities).  Subsequent  Fall  2009 
activities  included  the  development  of  marketing  focus,  student-centered  features.  A Faculty  Development  Guide 
was  developed  to  orient  faculty  to  working  with  Extended  Learning  and  goals,  features,  and  needs  of  the  IDT 
master’ s program. 

A Spring  2010  Faculty  Working  Group  will  be  convened  using  the  activity  analysis  discussion  list  and  the 
boundary  crossing  opportunities  table  to  discuss  program  features  and  development  challenges  and  decisions  to  be 
made.  Development  with  Extended  Learning  for  the  first  courses  will  occur  using  an  in-house  course  preparation 
checklist  and  the  Quality  Matters  rubric  (University  of  Maryland). 

Implications  for  Using  Activity  Theory 

Three  implications  of  activity  theory  for  E-Learning  can  be  characterizing  as  (a)  making  explicit,  (b) 
providing  opportunities,  and  (c)  sustaining  growth.  Activity  theory  makes  explicit  program  goals  and  brings  to  the 
surface  issues  and  concerns  for  the  different  constituents.  Using  activity  theory  as  both  analysis  and  synthesis  tool 
identifies  opportunities  for  conflict  to  be  addressed,  for  what  Engestrom  (2002)  calls  “expansive  learning.”  Whether 
the  scope  of  program  development  is  viewed  as  global  or  local,  top-down  or  bottom-up,  activity  theory  can  be  useful 
to  keep  discussion,  designing  and  trying  out,  and  revision,  a process  that  is  proactive,  scalable,  and  self-perpetuating, 
or  sustainable  embedding  as  characterized  by  Nichols  (2007). 

Guidelines  for  Using  Activity  Theory 

Based  on  what  has  been  accomplished  so  far  the  first  guideline  is  to  brief  as  many  participants  as  possible 
on  the  purpose  on  the  purpose  of  the  activity  systems.  In  an  academic  setting  this  activity  demonstrates  an  attempt  to 
use  theory  in  an  actual  setting. 

A second  guideline  would  be  to  specify  the  procedures,  although  these  may  have  to  be  adapted  to  the 
specific  context  of  the  program  development.  Jonassen,  Tessmer,  and  Hannum  (1999)  suggest  six  steps: 

1 . Clarify  the  purpose  for  the  activity  system 

2.  Provide  a big  picture  of  the  overall  initiative 

3.  Specify  the  activities  to  be  analyzed 

4.  Examine  the  role  of  the  tools 

5.  Address  the  internal  and  external  context 

6.  Monitor  what  is  happening  and  document  progress  and  the  process 

A third  guideline  is  to  collaborate  continually  and  frequently,  but  take  into  account  the  time  of  faculty, 
students,  and  administrators.  A fourth  guideline  stems  from  the  third;  namely,  to  share  the  results  of  the  activity 
system  approach  and  results  in  the  decision-making  process.  A fifth  guideline  would  be  to  document  your  activity 
and  to  report  its  results  to  participants. 

A worksheet  that  provides  the  activity  system  visual  as  an  analysis  organizer  can  be  found  at: 
http://web.me.com/nshambaugh/MediaCenterAJploaded_Files.html 
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Abstract 

Novice  and  expert  problem  solvers  use  different  strategies  in  approaching  ill-structured  problems,  indicating  lower- 
order  and  higher-order  thinking  skills  respectively.  There  is  a need  to  guide  novices  to  use  better  problem-solving 
strategies  through  developing  their  higher-order  thinking  skills.  According  to  theory  and  research,  analytic, 
generative,  and  evaluative  thinking  skills  are  essential  to  successful  ill-structured  problem  solving  performance.  We 
propose  a framework  designed  to  promote  higher-order  thinking  skills  during  ill-structured  problem-solving. 
Scaffolding  features,  types,  and  functions  of  an  open-ended  learning  environment  can  be  used  as  guidelines  for 
creating  scaffolds  to  facilitate  the  novices'  acquisition  and  utilization  of  the  three  fundamental  thinking  skills. 
Possible  ways  of  scaffolding  novice  problem  solvers  to  use  higher-order  thinking  skills  are  discussed  based  on  the 
scaffolding  design  in  various  ill-structured  learning  domains. 

Problem  Area 

The  real-world  problems  confronting  practitioners  are  usually  saturated  with  “complexity,  uncertainty, 
instability,  uniqueness,  and  value  conflict'’  and  require  the  practitioner  to  “choose  among  multiple  approaches  to 
practice  or  devise  his  own  way  of  combining  them”  (Schon,  1993,  pp.  17-18).  Cognitive  psychologists  define  these 
problems  as  ill-structured  or  ill-defined  because  they  have  unspecified  problem  space  (e.g.,  elements,  goals, 
constraints,  principles,  etc.),  divergent  perspectives,  multiple  solutions,  and  unclear  evaluation  criteria,  entailing  a 
dialectic  and  iterative  problem-solving  process  (Chi  & Glaser,  1985;  Kitchner,  1983;  Sinnott,  1989;  Voss  & Post, 
1989).  From  constructivism  and  situated  cognition  perspectives,  educational  researchers  (e.g.,  Bransford,  Brown,  & 
Cocking,  2000;  Bransford  & Stein,  1993;  Jonassen,  1997)  emphasize  the  importance  of  engaging  students  in  ill- 
structured  problem  solving  in  order  to  situate  learning  in  authentic  tasks  and  to  develop  tr  ansferable  knowledge  and 
skills. 

Ill-structured  problem  solving  is  difficult  for  novices  with  deficiencies  in  the  prerequisites  of  this  activity. 
Literature  show  that  the  major  predictors  of  ill-structured  problem-solving  performance  include:  1)  domain 
knowledge  (e.g.,  Chi  & Glaser,  1985;  Diakidoy,  2001;  Shin,  Jonassen,  & McGee,  2003),  2)  dispositions  and  beliefs 
(e.g.,  Mumford,  Baughman,  Threlfall,  & Uhlman,  1993;  Schraw,  Dunkle,  & Bendixen  1995),  3)  higher-order 
thinking  skills  (e.g.,  Lee  & Cho,  2007;  Mumford,  Baughman,  & Threlfall  et  al.,  1996,  1997;  Schunn,  McGregor,  & 
Saner,  2005;  Shin  et  al.,  2003),  and  4)  metacognition  (e.g..  Brown,  1987;  Lin,  2001;  Sinnott,  1989).  Novices  do  not 
possess  extensive  and  well-organized  knowledge  framework  for  problem  solving.  Many  of  them  develop  simple 
epistemological  beliefs  about  the  problem  domain  and  take  close-minded  attitudes  toward  new  ideas.  As  a result, 
they  tend  to  apply  a single  schema  to  ill-structured  problems,  preclude  exploration  of  alternative  perspectives,  and 
fail  to  effectively  evaluate  and  justify  the  proposed  solutions.  Also,  because  of  limited  working  memory,  few 
novices  are  able  to  consciously  regulate  their  cognitive  processing  during  problem  solving. 

To  address  these  challenges,  more  and  more  educational  researchers  have  realized  the  need  to  provide 
external  instructional  support  to  facilitate  cognitive  and  metacognitive  processes  during  ill-structured  problem 
solving.  A variety  of  scaffolds  have  been  implemented  to  support  the  aforementioned  four  predictors  by  facilitating 
knowledge  integration  (e.g.,  Chen  & Bradshaw,  2007;  Davis  & Linn,  2000),  argumentation  and  justification  (e.g., 
Cho  & Jonassen,  2002;  Oh  & Jonassen,  2007),  exploration  of  alternative  ideas  or  perspectives  (e.g.,  Choi  & Lee, 
2009;  Uribe,  Klein,  & Sullivan,  2003),  and  metacognitive  regulation  of  the  problem-solving  process  (e.g.,  Ge  & 
Land,  2003;  Wolf,  Brush,  & Saye,  2003).  According  to  the  findings,  although  learners’  ill-structured  problem- 
solving skills  were  enhanced  to  some  extent,  new  challenges  emerged  as  the  scaffolding  effects  were  unsatisfactory. 
For  example,  learners  tended  to  use  the  scaffolds  superficially  and  mindlessly;  not  all  learners  can  benefit  from  the 
same  scaffolds;  multiple  types  of  scaffolds  increased  cognitive  load.  This  paper  aims  to  develop  a theoretical 
framework  to  address  the  original  and  emerging  challenges  in  the  area  of  scaffolding  novices  to  solve  ill-structured 
problems,  with  a focus  on  three  particular  issues. 
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First,  novices  have  difficulty  applying  higher-order  thinking  skills  to  ill-structured  problem  solving.  They 
tend  to  analyze  the  complex  problems  in  simplified  ways,  overlooking  the  multiple  factors  and  representations 
(Carrithers,  Ling,  & Bean,  2008;  Powell  & Willemain,  2007;  Prietual  & March,  1991).  Moreover,  they  focus  on  the 
fragmented  details  instead  of  the  meanings  and  relationships  and  have  difficulty  distinguishing  the  relevant  from  the 
irrelevant  information  (Powell  & Willemain,  2007).  In  generating  solutions,  novices  usually  fail  to  consider 
alternative  ideas  and  perspectives  (Osana,  Tucker,  & Bennett,  2003;  Prietual  & March,  1991)  or  lose  their  focus  in 
discursive  exploration  of  ideas  (Kapur,  2008;  Powell  & Willemain,  2007).  They  are  likely  to  build  solutions  based 
on  available  salient  information  rather  than  consult  and  integrate  additional  resources.  Also,  it  is  difficult  for  novices 
to  determine  good  criteria  for  evaluating  the  solutions  and  developing  strong  arguments.  They  tend  to  make  rapid, 
uninformed  decisions,  which  in  turn  cut  short  their  search  for  alternatives  (Osana  et  al.,  2003;  Schunn  et  al.,  2005). 
Studies  showed  that  higher-order  thinking  skills,  which  are  supported  by  the  other  predicting  factors  mentioned 
earlier,  have  significant  influence  over  ill-structured  problem-solving  performance  (Lee  & Cho,  2007;  Mumford  et 
al.,  1996,  1997;  Schun  et  al.,  2005;  Shin  et  al.,  2003).  Inadequate  higher-order  thinking  skills  will  inevitably  limit 
the  solver’s  cognitive  capacities  of  processing  the  ill-structured  problems. 

Second,  learners  tend  to  misunderstand,  misuse,  or  ignore  the  scaffolds  and  their  functions,  reducing  the 
scaffolding  effectiveness.  Frequently,  learners  fail  to  reflect  on  the  purposes  and  strategic  functions  of  the  scaffolds 
and  rarely  go  beyond  answering  questions  to  seek  further  understanding  (e.g.,  Oliver  & Hannafin,  2000).  They  are 
inclined  to  use  lower-order  thinking  skills  in  responding  to  the  scaffolds  or  simply  ignore  the  opportunities  of 
higher-order  thinking  (e.g.,  Davis  & Linn,  2000;  Land  & Zembal-Saul,  2003).  They  try  to  solve  problems  with  their 
presuppositions  rather  than  follow  the  scaffolds  to  critically  reflect  on  their  own  and  others’  perspectives  (e.g.,  Choi 
& Lee,  2009;  Ge,  Chen,  & Davis,  2005).  Faced  with  confusions  or  difficulties,  they  easily  give  up  using  the 
scaffolds  that  demand  mental  efforts,  especially  when  they  experience  cognitive  overload  (e.g.,  Ge  et  al.,  2005; 
Greene  & Land,  2000).  These  problems  are  partly  due  to  the  learners'  lack  of  prior  knowledge,  simple 
epistemological  beliefs,  and  limited  higher-order  thinking  and  metacognitive  skills.  Additionally,  the  design  of  the 
scaffolds  may  also  be  responsible.  Scaffolds  that  cannot  be  easily  understood  by  the  learners  or  fail  to  support  their 
zones  of  proximal  development  are  not  likely  to  achieve  the  intended  effects. 

Third,  despite  the  positive  findings  from  experimental  studies,  the  learners’  skills  developed  under 
scaffolding  may  not  be  robust  enough  to  support  problem  solving  in  new  contexts.  Many  studies  examined 
scaffolding  effectiveness  based  on  the  comparison  between  experimental  and  control  groups  (e.g.,  Cho  & Jonassen, 
2002;  Davis  & Linn,  2000;  Ge  & Land.  2003;  Manlove,  Lazonder,  & de  Jong,  2007;  Oh  & Jonassen,  2007).  Criteria 
for  assessing  problem-solving  performance  were  aligned  with  the  scaffolded  skills  and  processes.  They  gave  the 
experimental  group  strong  advantages  that  may  have  confounded  the  conclusion  that  the  scaffolds  will  significantly 
improve  ill-structured  problem-solving  performance.  Moreover,  since  most  studies  did  not  fade  the  scaffolds,  there 
was  little  evidence  that  the  improved  performance  had  been  internalized  by  the  learners  and  could  be  transferred  to 
new  situations.  A few  studies  that  did  fade  the  scaffolds  found  partial  or  even  no  improvement,  which  might  be 
explained  by  the  limited  period  of  intervention  (Choi  & Lee,  2009)  and  inconsistencies  between  scaffolded  skills 
and  overall  problem-solving  performance  (Cho  & Jonassen,  2002;  Oh  & Jonassen,  2007). 

According  to  the  three  issues,  scaffolding  ill-structured  problem  solving  is  a pressing  yet  challenging  task, 
particularly  in  promoting  the  novices  to  use  higher-order  thinking  skills,  which  are  vital  for  professional 
development  but  difficult  to  obtain  without  facilitations.  Nevertheless,  as  external  supports,  scaffolding  may  not 
always  be  effective  and  can  easily  lead  to  lower-order  thinking  when  their  functions  are  not  recognized  or  utilized 
by  the  learners.  Even  though  progress  in  learners’  higher-order  thinking  and  problem-solving  skills  was  evidenced 
by  previous  studies,  it  may  only  occur  under  certain  conditions  and  cannot  be  sustained  or  guaranteed.  These  issues 
should  be  addressed  before  learners  can  truly  benefit  from  and  gain  control  over  ill-structured  problem-solving 
activities. 


Theoretical  Framework 

To  support  the  design  and  investigation  of  scaffolding  for  ill-structured  problem  solving,  a theoretical 
framework  should  be  developed  to  guide  the  inquiry.  Since  improving  higher-order  thinking  skills  is  a critical  issue, 
this  framework  aims  at  clarifying  the  specific  higher-order  thinking  skills  involved  in  ill-structured  problem  solving 
and  adapting  the  scaffolding  features,  types,  and  functions  to  promote  those  skills  (see  Figure  1). 
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Higher-Order  Thinking 


The  idea  of  higher-order  thinking  skills  has  no  clear  definition;  it  has,  however,  functioned  as  an  important 
reminder  that  there  are  more  advanced  thinking  skills  that  learners  should  develop  beyond  the  basic  cognitive  skills 
such  as  remembering,  comprehension,  and  application  (Bloom,  1956;  Ennis,  1987). 

Among  the  literature  on  higher-order  thinking  skills,  four  sources  of  seminal  works  stand  out,  including  Dewey’s 
reflective  thinking  (1933),  Bloom’s  taxonomy  of  educational  objectives  (1956),  Guilford's  structure-of-intellect 
model  (1967),  and  Ennis’s  critical  thinking  (1987).  These  theories  and  their  further  development  identified  specific 
higher-order  thinking  skills  from  different  perspectives  and  converged  on  the  point  that  higher-order  thinking  skills 
are  integral  to  problem-solving  skills.  Dewey  ( 1933)  stressed  the  idea  of  problem  solving  in  reflective  thinking  by 
highlighting  “a  state  of  doubt”  and  “an  act  of  inquiry  to  resolve  the  doubt”  (p.  12).  The  three  higher-order  objectives 
in  Bloom’s  taxonomy  (1956)  - analysis,  synthesis,  and  evaluation  - are  the  essential  components  of  what  he  called 
“intellectual  abilities  and  skills”  and  “problem-solving  skills”  (p.  39).  In  Guilford's  structure-of-intellect  model 
(1967),  the  three  higher-order  operations  - divergent  production,  convergent  production,  and  evaluation  - provided 
theoretical  ground  for  creativity  which  was  considered  as  a “discovered  problem-solving  process”  (Csikszentmihalyi 
& Getzels,  1971).  Ennis  (1987)  suggested  the  process  of  problem  solving,  which  integrated  thinking  into  procedural 
steps,  as  one  way  of  organizing  critical  thinking  to  decide  on  an  action  (p.  23).  Although  the  four  theories  come  from 
quite  different  fields,  including  philosophy,  assessment,  intelligence,  and  law,  their  shared  connection  with  problem 
solving  has  made  it  possible  to  identify  the  common  higher-order  thinking  categories  and  skills  as  listed  in  Table  1. 
Appendix  A shows  how  this  classification  scheme  was  identified  from  the  higher-order  thinking  skills  described  by 
each  theory. 


Figure  1.  Framework  of  scaffolding  higher-order  thinking  in  ill-structured  problem  solving 
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Table  1.  Classification  of  higher-order  thinking  skills 


Analytic  Thinking 

Identifying  relevant  facts,  evidence, 
and  information. 

Analyzing  given  statements,  reasons, 
hypotheses,  conclusions  etc. 

Identifying  unstated  reasons,  ideas, 
statements,  assumptions,  etc. 

Analyzing  the  relationships  and 
inexplicit  structures  of  the 
elements  and  parts. 


Generative  Thinking 

Producing  or  creating  plans,  ideas, 
details,  solutions,  products, 
communications,  etc.  that  are 
divergent  and  open-ended. 
Inducing  or  deducing  beliefs, 
hypotheses,  generalizations, 
conclusions,  value  judgments, 
etc.  that  are  convergent  and 
close-ended. 


Evaluative  Thinking 

Evaluating  the  identified  information 
or  generated  ideas  based  on 
correctness,  consistencies, 
reliability,  and  credibility. 
Evaluating  the  identified  information 
or  generated  ideas  based  on 
satisfaction  to  ultimate  goals  or 
appropriateness  to  the  context. 


The  three  major  categories  of  higher-order  thinking  skills  - analytic  thinking,  generative  thinking,  and 
evaluative  thinking  - are  interconnected  with  one  another  and  align  well  with  the  problem-solving  process.  First  of 
all,  analytic  thinking  serves  as  the  prerequisite  of  generative  and  evaluative  thinking.  It  builds  upon  lower-order 
skills  that  bring  to  bear  appropriate  principles  and  emphasizes  the  detection  of  elements,  parts,  and  their  connections 
or  organizations  (Bloom,  1956,  p.  144).  The  goal  is  to  clarify  and  focus  on  certain  issues  (Ennis,  1987,  p.  17)  and  to 
support  or  justify  the  generation  of  ideas  and  beliefs  (Dewey,  1933,  p.9).  Both  the  purpose  and  function  of  analytic 
thinking  are  consistent  with  the  identification  of  problems  and  the  search  for  ideas  to  generate  solutions. 

Generative  thinking  is  one  step  beyond  analytic  thinking  and  one  step  prior  to  evaluative  thinking.  With  the 
information  identified  from  analysis,  it  generates  ideas  in  two  different  ways:  divergent  production  and  convergent 
production  (Guilford,  1967).  The  former  generates  multiple,  varied,  and  elaborated  ideas  that  are  integrated  with 
individuals’  feelings  and  experience  (Bloom,  1956,  p.  169;  Guilford,  1956,  p.  138).  It  is  best  applied  to  open-ended 
situations  that  allow  for  multiple  answers  and  encourage  alternative  ideas.  The  latter  generates  only  one  or  very 
limited  numbers  of  ideas  through  logic-tight  inference  of  the  given  information  (Ennis,  1987;  Guilford,  1956).  It  is 
most  appropriate  for  situations  requiring  ideas  derived  from  rigorous  reasoning  based  on  evidence,  rules,  and 
principles.  The  two  types  of  generative  thinking  can  be  applied  selectively  to  problem  construction  and  solution 
generation,  depending  on  the  situation. 

Evaluative  thinking  assists  analytic  thinking  to  distinguish  the  relevant  and  credible  sources  of  information 
and  supports  generative  thinking  to  make  good  decisions.  The  relationship  between  generative  and  evaluative 
thinking,  however,  is  more  complicated.  Evaluation  criteria  should  be  applied  not  only  to  evaluate  the  generated 
ideas  but  also  to  guide  idea  generation  (Baer,  2003).  Inappropriate  use  of  criteria  would  either  stifle  divergent 
production  or  mislead  convergent  production  (Cropley,  2006).  Bloom  (1956)  emphasized  the  difference  between 
internal  and  external  criteria  (p.  186).  By  assessing  the  internal  correctness,  consistencies,  reliability,  and  credibility, 
evaluative  thinking  helps  making  critical  judgments  about  the  given  information,  deductions,  and  inductions  (Ennis, 
1987).  Moreover,  by  using  external  criteria  associated  with  the  context  and  ultimate  goals,  evaluative  thinking 
informs  the  generation  and  selection  of  appropriate  information  or  ideas  (Bloom,  1956;  Dewey,  1933;  Guilford, 
1967).  As  evaluative  thinking  is  inseparable  from  analytic  and  generative  thinking,  it  must  be  essential  to  problem 
construction  and  solution  generation  as  well. 

Theory  and  research  show  that  higher-order  thinking  skills,  as  represented  by  analytic,  generative,  and 
evaluative  thinking,  are  associated  with  domain  knowledge,  dispositions,  and  metacognition.  Domain  knowledge  is 
necessary  for  applying  general  thinking  principles  and  provides  the  sources  of  generative  thinking  (Dewey,  1933; 
Ennis,  1987;  Guildford,  1967).  Studies  found  that  in  ill-structured  situations,  domain  knowledge  facilitated  the 
thinking  process  of  structuralizing  and  reasoning  through  the  open-ended  problem  space  (e.g.,  Diakidoy  & 
Constantinou,  2001;  Lee  & Cho,  2007).  Some  advocates  claimed,  however,  that  teaching  higher-order  thinking  skills 
enhances  the  learning  of  domain  knowledge  as  well  (Dewey,  1933;  Walsh  & Paul,  1985).  Furthermore,  learners 
should  also  have  dispositions  that  encourage  them  to  use  higher-order  thinking  skills  (Dewey,  1933;  Ennis,  1987), 
such  as  being  open-minded,  seeking  clarifications,  focusing  on  the  main  point,  etc.  Studies  found  open-minded 
attitudes,  including  openness  to  new  ideas  and  flexibility  with  alternatives,  enhanced  ill-structured  problem-solving 
performance  (e.g.,  Mumford  et  al.,  1993;  Shin  et  al.,  2003),  presumably  through  activating  higher-order  thinking 
skills.  Base  on  this  connection,  there  is  the  possibility  that  intentional  use  of  higher-order  thinking  skills  will  help 
cultivate  the  learners'  dispositions.  In  addition,  studies  showed  that  the  ability  to  regulate  cognition  through  planning, 
monitoring,  and  evaluation  stimulated  learners’  conscious  consideration  of  multiple  goals,  constraints,  and  solutions 
in  ill-structured  problem  solving  (e.g.,  Reiter-Palmon,  Mumford,  O'Connor  Boes,  & Runco  1997;  Shin  et  al.,  2003). 
From  a theoretical  perspective.  Bloom  (1956)  argued  that  by  engaging  in  activities  requiring  complex  intellectual 
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abilities  and  skills,  learners  will  become  more  conscious  and  aware  of  the  cognitive  behavior  they  exhibit.  Similar  to 
domain  knowledge  and  dispositions,  metacognition  is  a contributor  to  higher-order  thinking,  and  the  operation  of 
higher-order  thinking  skills  may  in  turn  increase  learners’  metacognitive  consciousness  and  awareness. 

Higher-Order  Thinking  in  Ill-Structured  Problem  Solving 

As  ill-structured  problems  are  characterized  by  undefined  problem  space,  multiple  representations  and 
solutions,  and  unclear  evaluation  criteria  (Jonassen,  1997),  higher-order  thinking  skills  should  play  a pivotal  role  in 
resolving  these  uncertainties.  In  order  to  identify  the  roles  of  analytic,  generative,  and  evaluative  thinking  skills 
during  ill-structured  problem-solving  process,  four  types  of  ill-structured  problem-solving  models  that  perceive  ill- 
structuredness  from  different  perspectives  were  reviewed:  1)  classic  model  (Sinnott,  1989;  Voss,  1988),  2) 
instruction-oriented  model  (Jonassen,  1997),  3)  design  problem  solving  model  (Geol  & Pirolli,  1989,  1992),  and  4) 
creative  problem  solving  model  (Mumford,  Mobley,  Reiter-Palmon,  Uhlman,  & Doares,  1991;  Treffinger,  1995). 
Additionally,  studies  that  found  higher-order  thinking  skills  an  important  predictor  of  ill-structured  problem-solving 
performance  were  also  analyzed  for  corroboration  and  complementation.  Based  on  analysis,  the  analytic,  generative, 
and  evaluative  thinking  skills  required  during  problem  identification  and  solution  generation  were  derived  (see 
Appendix  B for  outline). 


Analytic  Thinking  in  Problem  Identification 

Analyzing  the  given  and  hidden  information  in  the  problem  space  is  the  first  step  of  problem  identification. 
The  problem  solver  should  identify  not  only  the  surface  level  elements  such  as  initial  state,  goals,  operators, 
constraints,  etc.  (Geol  & Pirolli,  1989;  Voss,  1988)  but  also  the  unstated  problems,  goals,  assumptions,  and 
perspectives  (Jonassen,  1997).  Relevant  schemata  or  categories  should  be  searched  and  retrieved  from  prior 
knowledge  to  encode  the  available  information  and  to  guide  further  data  finding  (Geol  & Pirolli,  1989;  Jonassen, 
1997;  Mumford  et  al.,  1991).  Lee  & Cho  (2007)  found  that  better  problem  finders  tend  not  to  confine  themselves  to 
the  given  information,  but  try  to  revive  maximum  knowledge  to  structuralize  the  situation  or  to  search  divergently 
for  new  information  beyond  what  is  suggested,  depending  on  how  ill-structured  the  problem  was.  In  addition,  the 
problem  solver  should  also  analyze  the  relationships  among  the  identified  elements,  such  as  causal  relationships, 
classifications,  and  knowledge-context  connections  (Jonassen,  1997;  Treffinger,  1995;  Voss,  1988). 

Generative  Thinking  in  Problem  Identification 

Proposing  new  operators  and  constructing  problem  representations  extend  analytic  thinking  by  generating 
new  meanings.  In  addition  to  encoding  information,  retrieved  schemata  or  categories  can  also  be  instantiated  into 
operators  with  the  contextual  information  to  compensate  for  the  missing  elements  in  the  problem  space  (Geol  & 
Pirolli,  1992).  When  the  perceived  problem  elements  point  to  one  unique  direction,  the  solver  can  represent  and 
solve  the  problem  through  convergent  production,  as  is  commonly  seen  in  experts'  problem-solving  activities  (Voss 
& Post,  1989).  However,  a more  typical  way  of  framing  ill-structured  problems  is  to  generate  multiple,  alternative 
representations  from  different  perspectives  and  converge  on  the  most  appropriate  representation  (Jonassen,  1997; 
Sinnott,  1989;  Treffinger,  1995;  Voss,  1988).  Studies  on  problem  construction  indicated  that  solvers  who  wrote 
down  all  possible  problem  statements  and  finally  reframed  the  problem  were  more  likely  to  develop  high-quality 
solutions,  especially  in  working  on  unfamiliar  problems  (e.g.,  Redmon,  Mumford,  & Teach,  1993;  Reiter-Palmon  et 
al.,  1997). 

Evaluative  Thinking  in  Problem  Identification 

Evaluative  thinking  assists  analytic  thinking  during  problem  analysis  and  assists  generative  thinking  during 
problem  construction.  To  efficiently  specify  the  problem  space,  the  solver  needs  to  attend  selectively  to  the  available 
information  and  the  activated  schemata  by  evaluating  their  internal  consistencies  and  relevance  to  goals  (Jonassen, 
1997;  Mumford  et  al.,  1991;  Sinnott,  1989).  Studies  found  that  the  problem  solver’s  abilities  to  distinguish  relevant 
from  irrelevant  information  and  to  select  best-fitting  concepts  or  categories  to  understand  the  problem  situation  are 
significant  predictors  of  solution  quality  and  originality  (e.g.,  Mumford  et  al.,  1996,  1997).  With  multiple  problem 
representations,  the  solver  also  needs  to  evaluate  and  select  among  the  competing  options  based  on  his  or  her 
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perceptions  and  ultimate  goals  (Jonassen,  1997;  Sinnott,  1989;  Treffinger,  1995),  and  at  times  to  negotiate  the 
problem  space  boundaries  based  on  the  constraints  (Geol  & Pirolli,  1992).  Although  there  is  no  clear  empirical 
evidence,  the  evolving  process  from  divergent  to  convergent  problem  representation  should  involve  evaluative 
thinking. 

Analytic  Thinking  in  Solution  Generation 

To  generate  high-quality  solutions,  the  solver  should  use  analytic  thinking  to  facilitate  ideation  and 
justification.  By  analyzing  links  among  the  selected  coding  categories  and  further  identifying  remotely  associated 
categories,  the  solver  will  be  inspired  to  generate  better  solution  ideas  (Mumford  et  al.,  1991;  Treffinger,  1995). 
Studies  on  ill-structured  problem-solving  strategies  found  that  when  solvers  were  suggested  to  analyze  the 
categories  of  potential  solutions  or  to  search  for  remotely  associated  categories  in  familiar  domains,  they  tended  to 
produce  more  high-quality  ideas  (e.g.,  Butler  & Kline,  1998;  Stoyanov  & Kirschner,  2007).  Moreover,  for  large-size 
problems,  the  solver  should  decompose  the  entire  problem  into  components  and  analyze  their  connections  so  as  to 
generate  partial  solutions  and  combine  them  at  a later  stage  (Geol  & Pirolli,  1992).  Additionally,  after  generating 
solutions,  it  is  important  for  the  solver  to  justify  the  preferred  solutions  by  identifying  relevant  facts,  evidence, 
statements,  and  conjectures  to  warrant  their  claims  (Jonassen,  1997;  Voss  & Post,  1989),  the  analysis  of  which  is 
essential  to  ill-structured  problem  solving  (Shin  et  al.,  2003). 


Generative  Thinking  in  Solution  Generation 

All  the  previous  higher-order  thinking  activities  are  preparing  for  applying  generative  thinking  to  develop 
solutions.  For  highly  ill-structured  problems  with  multiple  representations,  there  is  the  need  to  generate  multiple, 
alternative  solutions  through  divergent  production  (Jonassen,  1997;  Sinnott,  1989;  Treffinger,  1995;  Voss  & Post, 
1989).  Although  it  is  difficult  to  generate  fluent  solution  ideas  to  complex  problems,  especially  for  novices 
(Jonassen,  1997),  two  empirically-validated  strategies  might  be  used.  First,  through  combining  and  reorganizing  the 
categories  analyzed  in  the  previous  stage,  the  solver  may  break  the  fixed  paradigms  and  come  up  with  new 
categories  that  support  solution  generation  (Butler  & Kline,  1998;  Mumford  et  al.,  1991;  Stoyanov  & Kirschner, 
2007).  Second,  the  solver  may  also  generate  multiple,  better  solutions  by  considering  the  perspectives  of  different 
stakeholders  involved  in  the  problem  (Butler  & Kline,  1998;  Choi  & Lee,  2009;  Jonassen,  1997).  However, 
convergent  production  is  more  effective  for  generating  one  unique  or  relatively  fewer  solutions  in  problem  situations 
with  specific  representations  (Diakidoy  et  al.,  2001;  Voss  & Post,  1989).  Convergent  production  should  also  be  used 
during  and  after  divergent  production,  as  the  problem-solving  process  evolves  from  exploring  divergent  ideas  to 
converging  on  the  promising  solutions  (Jaarsveld  et  al.,  2005),  constructing  evidence-based  arguments  to  justify 
decisions  (Jonassen,  1997;  Shin  et  al.,  2003),  and  incrementally  adding  details  to  refine  the  final  solution  (Geol  & 
Pirolli,  1992). 

Evaluative  Thinking  in  Solution  Generation 

Evaluative  thinking  in  solution  generation  has  both  formative  and  summative  natures.  From  the  formative 
sense,  the  solver  should  evaluate  the  interim  solution  ideas  while  proposing  and  refining  them  to  satisfy  the  goals  at 
hand  and  to  stay  within  the  constraints  (Baer,  2003;  Mumford  et  al.,  1991).  Divergent  production  using  the  two 
strategies  mentioned  earlier  involves  evaluative  thinking,  as  the  solver  needs  to  generate  ideas  fitting  certain 
categories  or  perspectives,  leading  to  more  meaningful  solutions  (Butler  & Kline,  1998;  Stoyanov  & Kirschner, 
2007).  Also,  design  problem  solving  study  found  that  the  incremental  design  process  was  integrated  with  generation, 
evaluation,  and  modification  of  design  components  based  on  contextual  factors  (Geol  & Pirolli,  1992).  From  the 
summative  sense,  the  solver  has  to  either  select  from  or  narrow  down  the  multiple  solutions  by  evaluating  them  with 
explicit  internal  and  external  criteria  (Jonassen,  1997;  Sinnott,  1989;  Treffinger,  1995;  Voss,  1988).  While  moving 
from  divergent  to  convergent  production  and  justifications,  good  problem  solvers  weigh  the  solution  alternatives  and 
their  potential  outcomes  against  the  internal  constraints  and  ultimate  goals  to  decide  on  the  most  defensible  and 
appropriate  solutions  (Jaarsveld  & van  Leeuwenb,  2005;  Shin  et  al.,  2003). 

As  can  be  seen,  analytic,  generative,  and  evaluative  thinking  skills  each  plays  a unique  role  in  problem 
identification  and  solution  generation.  At  the  same  time,  they  are  inseparable  and  well-integrated  into  the  ill- 
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structured  problem-solving  process.  Since  the  sub  skills  involved  in  the  three  types  of  higher-order  thinking  are  not 
used  sequentially,  intelligent  problem  solvers  will  apply  them  flexibly  by  switching  to  proper  skills  within  or  across 
the  categories  and  phases  based  on  the  situation. 

Scaffolding  Higher-Order  Thinking  in  Ill-Structured  Problem  Solving 

After  instantiating  the  higher-order  thinking  skills  in  ill-structured  problem  solving,  the  most  important  task 
is  to  use  theory  and  research  on  scaffolding  to  develop  those  skills  in  novices  so  as  to  improve  their  problem-solving 
performance.  How  can  we  use  scaffolding  features,  types,  and  functions  to  enhance  higher-order  thinking  skills? 
How  can  we  design  the  scaffolds  to  address  the  challenges  reported  by  previous  studies?  How  can  we  help  learners 
sustain  and  transfer  the  skills  developed  under  scaffolding?  These  questions  could  be  best  answered  by  design-based 
research  (Barab  & Squire,  2004;  Brown,  1992).  As  a beginning,  this  framework  will  present  some  preliminary 
thoughts  based  on  existing  theory  and  research. 

Scaffolding  Features 

Scaffolding  was  originally  defined  as  an  adult  or  expert  interacts  with  a child  or  novice  to  help  him  or  her 
achieve  a goal  beyond  his  or  her  unassisted  efforts  (Wood,  Bruner,  & Ross,  1976).  This  metaphor  has  been 
generalized  to  structures  that  interact  with  novice  learners  to  help  them  accomplish  tasks  beyond  their  competencies 
(Puntambekar  & Hubscher,  2005;  Reiser,  2004).  According  to  Puntambekar  and  Hubscher  (2005),  the  current  notion 
of  scaffolding  has  the  following  features:  shared  understanding,  scaffolder,  ongoing  diagnosis,  calibrated  support, 
and  fading.  These  features  can  delineate  the  scaffolding  for  higher-order  thinking  during  ill-structured  problem 
solving  at  a macro  level. 

Shared  understanding  between  the  learners  and  the  scaffolder  regarding  the  goal  of  the  activity  is  critical 
for  the  learners  to  gain  ownership  of  the  task  (Puntambekar  & Hubscher,  2005).  In  the  current  context,  the  learners 
and  the  scaffolder  should  have  a common  understanding  of  solving  ill -structured  problems  with  higher-order 
thinking  skills.  Real-world  problem  situations  will  establish  the  overarching  goal  of  problem  solving  in  the  learning 
environment.  Moreover,  simplified  activities  of  applying  higher-order  thinking  to  problem  solving  might  be  used  for 
preparing  learners  with  shared  understanding  (e.g.,  Kolodner,  Crismond,  & Fasse  et  al.,  2003;  Reiser,  Tabak,  & 
Sandoval  et  al.,  2001). 

The  role  of  scaffolder  has  been  extended  from  more  knowledgeable  others  to  include  technology  tools  and 
resources,  advancing  the  techniques  of  providing  instructional  supports  (Lajoie,  2005;  Puntambekar  & Hubscher, 
2005).  Technology  and  human  scaffolds  are  mutually  beneficial  because  the  technology  scaffolder  can  provide 
routine  supports  to  common  learning  needs,  allowing  time  and  efforts  for  the  human  scaffolder  to  provide  on- 
demand,  customized  support  that  may  not  be  easily  provided  by  the  technology  scaffolder  (Saye  & Brush,  2002; 
Sharma  & Hannafin,  2007).  Because  of  the  complex  learning  involved  in  higher-order  thinking  and  ill-structured 
problem  solving,  “mixed  initiative”  designs  based  on  the  synergy  of  technology  and  human  supports  might  be 
particularly  helpful  (Pea,  2004,  p.  444). 

The  process  of  scaffolding,  including  ongoing  diagnosis,  calibrated  support,  and  fading  are  essential  yet 
largely  overlooked  in  current  design  and  implementation  of  scaffolding  (Pea,  2004;  Puntambekar  & Hubscher, 

2005).  The  scaffolder  should  monitor  the  learners’  current  levels  of  understanding  in  order  to  provide  “graduated 
assistance”  (Stone,  1998)  and  to  gradually  fade  the  scaffolds  to  let  the  learners  takeover  the  responsibility,  which  is  a 
process  of  “internalization”  (Vygotsky,  1978).  To  compensate  the  technological  limitations  in  dynamically 
calibrating  and  fading  supports,  redundancies  in  technology  scaffolds  can  provide  graduated  assistance 
(Puntambekar  & Hubscher,  2005),  and  human  scaffolders  can  assess  learners’  performance  to  adjust  the  levels  of 
both  human  and  technology  scaffolding  (Pea,  2004).  Therefore,  in  designing  mixed  scaffolds  for  the  current  context, 
technology  may  create  multiple  forms  and  levels  of  scaffolding  to  meet  learners’  developmental  needs  in  higher- 
order  thinking  and  ill-structured  problem  solving,  and  the  human  scaffolders  (e.g.,  instructor,  peers)  may  select  and 
fade  the  scaffolds  based  on  ongoing  diagnosis  of  the  learners'  progress. 

Scaffolding  Types  and  Functions 

According  to  the  literature,  scaffolding  types  include  hard,  fixed  scaffolds  and  soft,  dynamic  scaffolds 
(Saye  & Brush,  2002;  Wang  & Hannafin,  2008).  The  former  are  planned  in  advance  based  on  the  cognitive  activities 
involved  in  the  task  and  the  difficulties  learners  might  encounter.  They  are  usually  explicit,  non-negotiable,  and 
mediated  by  technology  to  constrain  learners’  actions  (Sharma  & Hannafin,  2007).  The  latter  are  created 
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dynamically  within  certain  situations.  They  are  more  customized  and  adaptive  to  individual  learners,  mainly 
provided  by  instructors,  experts,  or  peers  through  dialogic  interactions  (Saye  & Brush,  2002).  The  two  types  of 
scaffolds  parallel  the  two  complimentary  scaffolders  - technology  and  human  - mentioned  earlier. 

Hannafin,  Land,  and  Oliver  (1999)  classified  scaffolds  into  four  functions  that  support  student  learning  in 
open-ended  learning  environments:  conceptual  (what  knowledge  to  consider),  metacognitive  (how  to  think  about  the 
problem ),  procedural  (how  to  use  learning  environment  features),  and  strategic  (what  are  the  alternative  strategies) 
scaffolds.  The  procedural  scaffolds  were  later  defined  as  operational  steps  or  cognitive  structures  for  helping 
students  complete  a task  (Sharma  & Hannafin,  2007;  Wang  & Hannafin,  2008).  The  above  two  types  and  four 
functions  of  scaffolding,  which  emphasize  the  ways  and  purposes  of  creating  scaffolds,  can  guide  the  micro  level 
conceptualization  of  instructional  support  for  enhancing  the  three  higher-order  thinking  skills  during  ill-structured 
problem  solving. 

Conceptual  scaffolds.  They  help  learners  identify  conceptual  knowledge  related  to  a problem  and  create 
structures  of  conceptual  organization  (Hannafin  et  al.,  1999).  These  functions  are  consistent  with  analytic  thinking, 
as  the  solver  has  to  retrieve  relevant  schema  or  categories  from  prior  knowledge  for  encoding  information  and  to 
figure  out  the  organization  among  the  problem  space  elements,  sub  problems,  solution  categories,  and  evidence  of 
claims.  Hard  conceptual  scaffolds  such  as  links  to  available  resources,  concept  maps,  and  relationship  prompts  can 
facilitate  learners’  knowledge  retrieval  and  problem  analysis.  Chen  & Bradshaw  (2007)  and  Oliver  & Hannafin 
(2000)  used  advanced  organizers  to  help  students  consider  the  associated  knowledge  concepts  and  relationships  in 
solving  problems.  They  both  found  the  students  were  able  to  identify  and  analyze  the  relevant  knowledge  but  the 
younger  learners  have  difficulty  integrating  them  into  the  problem.  Soft  scaffolds  in  the  form  of  individualized 
suggestions,  feedback,  or  questioning  on  concepts  selection  and  structure  analysis  may  further  learners’ 
understanding  of  knowledge  and  its  applications  in  problem  contexts. 

Procedural  scaffolds.  They  provide  operational  sequences  of  thinking  or  completing  a task  and  use 
structures  to  focus  and  sustain  learners’  activities  (Sharma  & Hannafin,  2007;  Wang  & Hannafin,  2008).  Ill- 
structured  problem  solving  contains  general  procedures  of  using  analytic,  generative,  and  evaluative  thinking  to 
identify  problems  and  generate  solutions.  Procedural  scaffolding  can  guide  learners  through  the  problem-solving 
process  integrated  with  different  types  of  thinking,  providing  the  basis  for  them  to  internalize  and  self-regulate  those 
thinking  skills.  Hard  scaffolds  such  as  question  prompts  and  sentence  starters  were  commonly  used  for  procedural 
scaffolding.  However,  they  are  usually  insufficient  as  they  may  lead  to  a piecemeal  approach  to  learning  (Davis  & 
Linn,  2000)  or  induce  dependence  (Oliver  & Hannafin,  2000),  and  learners  with  different  competencies  have 
different  needs  for  procedural  guidance  (Ge  et  al.,  2005).  Soft  scaffolds  such  as  instructor-student  interactions 
should  not  only  attend  to  the  emergent  needs  for  procedural  support  (Wang  & Hannafin,  2008)  but  also  monitor  the 
learners’  progress  and  adjust  or  fade  the  hard  scaffolds  at  appropriate  times  to  encourage  internalization. 

Metacognitive  scaffolds.  They  guide  the  ways  to  think  about  the  problem  and  remind  the  learners  to  reflect 
on  the  goals  and  monitor  the  learning  process  (Hannafin  et  al.,  1999).  In  order  to  use  analytic,  generative,  and 
evaluative  thinking  skills  purposefully  and  effectively,  novice  problem  solvers  need  metacognitive  scaffolds  to  help 
them  understand  the  role  of  each  type  of  thinking  at  different  problem-solving  stages  and  consciously  use  those 
thinking  skills  through  planning,  monitoring,  and  evaluation,  until  they  achieve  automation.  Hard  metacognitive 
scaffolds  such  as  question  prompts  and  hints  have  been  frequently  integrated  into  procedural  scaffolds  to  help 
students  regulate  their  problem-solving  process  (e.g.,  Davis  & Linn,  2000;  Ge  et  al.,  2005).  However,  studies 
showed  that  learners  may  experience  difficulty  or  show  reluctance  in  using  metacognitive  scaffolds,  which  are  less 
prescriptive  and  require  more  efforts  to  grasp  (e.g.,  Oliver  & Hannafin,  2000;  Manlove  et  al.,  2007).  Soft  scaffolds 
based  on  ongoing  diagnosis  of  learners’  performance  are  more  likely  to  improve  achievements  (e.g.,  Azevedo, 
Cromley,  Winters,  Moos,  & Greene,  2005;  Greene  & Land,  2000)  and  may  serve  as  “metascaffolding”  of  how  to  use 
the  hard  scaffolds  (Pea,  2004). 

Strategic  scaffolds.  They  suggest  alternative  approaches  to  a problem  or  task  and  help  learners  identify  and 
evaluate  needed  information,  resources,  or  tactics  (Hannafin  et  al.,  1999;  Wang  & Hannafin,  2008).  Strategic 
scaffolds  may  be  particularly  useful  for  generative  thinking,  as  they  expose  the  learners  to  alternative  perspectives  or 
solutions  and  provide  them  tactics  for  divergent  and  convergent  production.  It  may  also  assist  analytic  thinking  by 
recommending  useful  recourses  and  support  evaluative  thinking  by  suggesting  important  evaluation  criteria.  Choi  & 
Lee  (2009)  provided  multiple  experts’  solutions  to  facilitate  problem  solving  and  found  the  learners  began  to 
acknowledge  the  different  perspectives  but  still  failed  to  evaluate  and  integrate  them  to  build  their  own  perspective. 
Oliver  & Hannafin  (2000)  used  strategic  scaffolds  for  helping  learners  evaluate  solutions  generated  from  divergent 
production,  but  most  students  glossed  over  those  scaffolds  by  selecting  a predetermined  solution.  Soft  scaffolds 
might  be  used  to  draw  learners’  attention  to  appropriate  use  of  the  hard  strategic  scaffolds  as  well  as  to  provide 
dynamic,  individualized  strategic  scaffolds  that  are  tailored  to  each  learner. 
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Implications 


This  framework  is  designed  to  guide  the  inquiry  of  scaffolding  ill-structured  problem  solving  with  a focus 
on  1)  learners’  difficulties  in  performing  higher-order  thinking,  2)  their  misunderstanding  and  misuse  of  the 
scaffolds,  and  3)  unsustainable  or  non-transferable  problem-solving  skills.  The  higher-order  thinking  skills  and 
scaffolding  features,  types,  and  functions  identified  in  the  framework  have  the  potential  to  address  these  major 
issues. 

Learners’  vague  ideas  about  the  types  of  thinking  required  during  ill-structured  problem  solving  may  lead 
to  lower-order  thinking.  By  situating  analytic,  generative,  and  evaluative  thinking  skills  in  ill-structured  problem 
solving  and  providing  procedural  and  metacognitive  scaffolds,  the  learners  will  be  able  to  understand  why  and  how 
to  apply  the  specific  skills  and  to  learn  to  actively  regulate  their  performance.  Conceptual  and  strategic  scaffolds 
may  facilitate  analytic  and  generative  thinking  in  particular  through  activating  the  learners'  prior  knowledge  and 
offering  resources  or  tactics.  Additionally,  guided  higher-order  thinking  may  help  cultivate  desirable  dispositions 
and  beliefs  in  the  long  term. 

According  to  previous  studies,  the  scaffolds  misunderstood  or  used  superficially  by  the  learners  were 
usually  hard,  static  scaffolds.  In  order  to  help  learners  recognize  the  scaffolds’  strategic  functions  and  benefit  from 
more  flexible  scaffolding,  it  would  be  helpful  to  integrate  soft,  dynamic  scaffolds  provided  by  human  tutors  or  peers. 
Scaffolding  in  the  form  of  social  interactions  will  not  only  provide  adaptive  supports  based  on  ongoing  diagnosis  of 
the  learners’  level  of  understanding,  but  also  promote  the  learners’  appropriate  use  of  hard  scaffolds  through 
“metascaffolding”  (Pea,  2004). 

Gradually  fading  the  scaffolds  is  necessary  to  develop  learners’  sustainable  problem-solving  skills  and  can 
be  best  achieved  by  mixed  scaffolding  design.  Although  technology-based  hard  scaffolds  may  not  fade  intelligently, 
the  human  scaffolder  can  accomplish  this  task  by  adjusting  the  functions  or  levels  of  hard  scaffolds  to  fit  the 
learners’  current  zone  of  proximal  development.  Moreover,  problem-solving  skills  developed  within  this  framework 
may  have  better  transferability,  because  as  the  essential  components  of  ill-structured  problem-solving  competencies, 
the  analytic,  generative,  and  evaluative  thinking  skills  can  be  generalized  and  reused  for  different  problem  contexts. 

Although  this  framework  is  grounded  in  theory  and  research  on  higher-order  thinking,  ill-structured 
problem  solving,  and  scaffolding,  there  is  still  much  unknown  about  the  synergy  of  the  three  theoretical  perspectives 
and  their  contribution  to  facilitating  novices’  ill-structured  problem-solving  activities.  Therefore,  empirical  studies 
should  be  conducted  in  ill-structured  domains  to  validate  and  refine  this  framework. 
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Abstract 

Computers  and  Internet  have  become  widely  available  to  teachers  and  students,  yet  tomorrow’s  teachers  are  not  well 
prepared  to  effectively  use  technology  in  teaching.  The  author  designed  problem-based  case  scenarios  and  divergent 
and  critical  thinking  scaffolds  to  guide  preservice  teachers’  learning  to  teach  with  technology.  The  study  conducted 
an  in-depth  investigation  of  the  preservice  teachers’  (n=4)  experience  and  perceptions  about  this  guided  problem 
solving  approach  by  analyzing  interview  transcriptions  and  responses  to  scaffolds  with  grounded  theory  methods. 
Facilitated  by  the  scaffolds,  the  preservice  teachers  used  divergent  and  critical  thinking  skills  along  with  their  own 
strategies  in  identifying  instructional  problems,  generating  and  evaluating  solutions,  and  developing  lesson  plans 
integrated  with  technology.  However,  their  personal  history  based  assumptions  and  strategies  tend  to  lead  them 
toward  convergent  and  uncritical  thinking.  Finally,  the  author  discussed  the  ill-structured  nature  of  technology 
integration  problems,  the  preservice  teachers’  characteristics  in  learning,  and  the  value  of  the  guided  problem 
solving  approach  in  preparing  tomorrow’s  teachers  to  use  technology. 

Introduction 

Well,  I had  no  idea  what  Web  2.0  tool  was.  I knew  I guess  Internet  sites  like  Myspace  and  Facebook.  I 
knew  they  were  things  out  there  but  I didn’t  know  how  prevalent  they  were  in  the  education  department.  I 
didn't  realize  the  resources  that  were  available  to  education  majors  or  educators  in  general.  Think  Finity 
was  one  that  we  use  that  I would  definitely  be  interested  in  using  it  in  the  future.  The  Delicious  Account  we 
set  up,  the  Social  Book  marking  we  did,  the  Inspiration  things  we  did,  they  were  all  great  resources  as  [for] 
an  educator.  And  many  of  them  could  be  used  for  the  students  as  well.  And  I look  forward  to  using  those  in 
the  classroom.  (Jason) 

Jason  kept  naming  different  technology  tools  when  talking  about  what  he  learned  on  the  Educational 
Technology  course.  His  comments  not  only  reflected  a beginning  preservice  teacher’s  excitement  about  the  variety 
of  available  technologies  but  also  indicated  his  interests  of  applying  those  tools  in  his  future  class.  However,  despite 
the  abundant  technology  resources  and  the  preservice  teachers’  enthusiasm,  research  shows  that  most  preservice 
teachers  were  not  receiving  adequate  preparation  of  teaching  with  technology  before  entering  the  field  (CEO  Forum, 
2001;  Yildirm,  2000;  Kay,  2006).  Single  educational  technology  courses  usually  focus  more  on  training  technology 
skills  detached  from  teaching.  Method  courses  have  not  been  well  integrated  with  technology  because  many  faculty 
members  lack  technology  skills  and  cannot  provide  good  examples.  Collaborations  between  preservice  and  inservice 
teachers  are  promising  but  tend  to  be  constrained  by  administrative  issues.  As  a result,  most  preservice  teachers  are 
not  well  prepared  to  teach  with  technology  when  they  begin  their  teaching  career.  Even  though  they  are  more 
comfortable  with  technologies,  they  use  it  less  than  experienced  teachers  in  delivering  instruction  or  engaging 
students  in  learning  activities  (Russell,  Bebell,  O'Dwyer,  & O'Connor,  2003). 

How  can  teacher  education  programs  prepare  preservice  teachers  to  use  technology  effectively  in  their  future 
classroom?  Researchers  suggest  that  preservice  teachers  should  be  provided  with  high  level  learning  opportunities 
that  address  authentic  pedagogical  problems  (Mishra  & Koehler,  2003).  They  should  not  only  be  exposed  to 
examples  of  classroom  use  of  technology  (Russell  et  al.,  2003)  but  also  engage  in  hands-on  activities  to  develop 
their  technology  integration  skills  by  constructing  and  implementing  their  own  lessons  (Dawson,  Pringle,  & Adams, 
2003). 

Informed  by  researchers’  suggestions,  this  study  was  focused  on  facilitating  preservice  teachers'  learning  to 
teach  with  technology  through  a guided  problem  solving  approach.  Case  scenarios,  embedded  with  instructional 
problems  that  teachers  may  encounter  in  real  classroom  settings,  were  created  to  engage  preservice  teachers  in  an 
authentic-oriented  learning  environment.  As  the  essence  of  this  approach,  problem-solving  scaffolds  with  an 
emphasis  on  divergent  and  critical  thinking  were  designed  to  guide  preservice  teachers  in  identifying  the  ill- 
structured  instructional  problems  and  generating  technology-integrated  lesson  plans  and  artifacts  as  solutions.  The 
purpose  of  this  study  was  to  understand  the  experience  and  perceptions  of  the  preservice  teachers  in  learning 
technology  integration  through  this  approach,  as  well  as  to  identify  the  issues  and  challenges  need  to  be  addressed  in 


426 


future  implementation.  Since  little  is  known  about  using  guided  problem  solving  as  a strategy  for  preparing 
preservice  teachers  to  teach  with  technology,  findings  from  this  study  will  inform  the  design  of  the  scaffolds  and 
contribute  to  the  knowledge  of  researchers  and  teacher  educators  in  the  field  of  preparing  tomorrow’s  teachers  to  use 
technology.  Three  research  questions  were  posed  in  this  study: 

1 . What  are  the  roles  of  the  scaffolds  in  facilitating  preservice  teachers’  problem-solving? 

2.  What  strategies  do  preservice  teachers  use  during  the  problem-solving  process? 

3.  How  do  preservice  teachers  perceive  their  problem-solving  experience? 

4. 

Theoretical  Framework 

Technology  Integration  as  an  Ill-structured  Problem 

Teaching  is  an  ill-structured  discipline  characterized  by  the  complex  concepts  and  principles,  the  dynamic 
nature  of  the  situations,  and  the  variability  of  features  across  cases  (Mishra,  Spiro  & Feltovich,  1996;  Spiro  & Jehng, 
1990).  Integrating  technology  into  teaching  further  complicates  the  domain,  because  technology  brings 
unprecedented  opportunities,  affordances,  and  constraints  associated  with  teaching  content  and  pedagogy,  thus 
expanding  the  problem  space  of  teaching.  Designing  and  implementing  technology  integrated  lesson  is  an  ill- 
structured  problem-solving  process  that  represents  a special  challenge  to  knowledge  application  in  the  teaching 
domain. 

The  typical  characteristics  of  ill-structured  problems,  including  the  lack  of  prototypic  cases,  ill-defined 
elements,  multiple  domains  and  solutions  (Jonassen,  1997),  are  evident  in  technology  integration.  First  of  all, 
technology  integration  is  always  situated  in  a context  with  specific  subject  matter,  grade  level,  student  backgrounds, 
and  available  technologies,  etc  (Mishra  & Koehler,  2006).  The  way  of  incorporating  technology,  pedagogy,  and 
content  in  teaching  vary  from  case  to  case.  Also,  many  aspects  of  technology  integration  problems  are  not  well 
specified  for  the  teacher,  such  as  the  goal  state  of  using  technology,  the  appropriate  pedagogies,  the  available  tools 
and  their  functions,  the  potential  constraints,  etc.  It  is  clear  that  the  teacher,  as  the  problem  solver,  needs  to  clarify 
the  situation  before  trying  to  solve  the  problem.  As  suggested  by  the  Mishra  and  Koehler’s  (2006)  technology, 
pedagogy,  and  content  knowledge  (TPCK)  framework,  teaching  with  technology  requires  the  integration  of  three 
knowledge  domains  - technology,  pedagogy,  and  content  - as  well  as  the  complex  interplay  among  them. 
Technology  is  not  something  add-on  to  pedagogy  and  content  but  can  raise  fundamental  questions  and  initiate 
reconfiguration  of  the  three  components.  Most  importantly,  there  is  no  single  right  solution  to  technology  integration 
problems  (Mishra  & Koehler,  2006).  Ways  of  using  technology  in  teaching  are  divergent  rather  than  convergent  in  a 
given  situation,  which  is  the  same  for  solutions  to  any  ill-structured  problems.  The  teacher  should  consider  a variety 
of  solutions  and  make  rational  decisions  based  on  their  judgments  about  the  context. 

Divergent  and  Critical  Thinking  in  Ill-Structured  Problem-Solving 

Runco  (2003)  views  problem-solving  as  a creative  critical  process,  during  which  creative  thinking  helps 
generate  ideas  for  solving  the  problem,  while  critical  thinking  helps  to  evaluate  those  ideas  and  select  the  best 
solution.  With  a close  examination  of  both  the  creative  and  critical  sides  of  problem-solving,  the  essential  thinking 
skills  can  be  identified.  The  creative  side  of  problem-solving  is  centered  by  divergent  thinking,  which  involves 
thinking  of  many  ideas  (fluency),  varied  ideas  (flexibility),  new  ideas  (originality),  and  adding  details  to  improve  the 
ideas  (elaboration)  (Guilford,  1967;  Basadur  et  al.,  1986).  The  critical  side  of  problem-solving  includes  critical 
thinking  skills  such  as  clarifying  the  situation,  making  inferences  with  available  information,  evaluating  sources, 
assumptions,  arguments,  and  ideas,  and  monitoring  the  thinking  process  and  outcomes  (Bruning  et  al.,  2004;  Ennis, 
1987;  Halpern,  1997;  Perkins,  1986).  Thinking  skills  on  both  sides  play  a significant  role  in  ill-structured  problem 
solving  such  as  technology  integration,  which  is  an  instructional  design  process  that  requires  a design  structure 
based  on  diversified  perspectives  as  well  as  multiple  solution  ideas  and  evaluation  criteria  for  reaching  the  best 
solution  (Jonassen,  1997;  Jonassen,  2000). 

Preparing  Preservice  Teachers  for  Technology  Integration 

With  respect  to  the  ill-structured  domain  of  technology  integration,  teacher  educators  and  researchers  realize 
that  preservice  education  is  a natural  and  important  place  to  start  preparing  teaching  candidates  with  necessary  skills 
(Kay,  2006).  Various  strategies  have  been  created  for  developing  preservice  teachers’  technology  integration  skills, 
such  as  faculty  modeling  in  method  classes  (e.g.  Vannatta  & Beyerbach,  2000),  collaboration  with  mentor  teachers 
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(e.g.  Thompson  et  al.,  2003),  incorporating  technology  in  field  teaching  (e.g.  Brush  et  al.,  2003).  However,  evidence 
suggests  that  teacher  education  programs  have  not  been  successful  in  preparing  new  teachers  to  use  technology 
effectively  in  teaching  (CEO  Forum,  2001;  Yildirm,  2000;  Kay,  2006).  One  of  the  major  obstacles  is  that  preservice 
teachers  lack  opportunities  and  supports  to  inquire  into  technology  integration  by  designing  and  implementing 
lessons  for  relevant  teaching  situations  (Dawson,  Pringle,  & Adams,  2003).  In  other  words,  they  are  not  receiving 
enough  training  for  solving  ill-structured  technology  integration  problems,  which  makes  them  feel  unprepared  when 
beginning  their  teaching  career. 

However,  the  complexity  of  technology  integration  problems  poses  challenges  to  preservice  teachers  who  are 
novices  in  teaching.  As  discussed  previously,  successful  technology  integration  requires  divergent  and  critical 
thinking  skills,  systematic  problem-solving  skills,  and  integrated  knowledge  of  technology,  pedagogy,  and  content. 
Research  on  preservice  teachers'  learning  shows  that  they  come  to  teacher  education  programs  with  powerful, 
personal  history  based  theories  about  good  teaching,  preventing  them  from  thinking  critically  about  their 
assumptions  (Knowles  & Holt-Reynolds,  1991;  Schwarz  & Gwekwerere,  2006).  Also,  they  recall  self  experience  as 
a student  as  a base  for  generalizing  beliefs  about  students'  learning  as  well  as  modeling  the  learning  experience  they 
want  to  provide  in  their  classroom  (Holt-Reynolds,  1992).  This  tendency  may  conflict  with  the  desired  skill  of 
thinking  divergently  to  consider  different  perspectives  and  alternative  teaching  strategies.  Moreover,  although  it  may 
not  be  difficult  for  preservice  teachers  to  acquire  technology  skills,  they  are  not  likely  to  have  sufficient 
technological  pedagogical  content  knowledge,  which  should  be  developed  from  authentic  professional  practices 
(Mishra  & Koehler,  2006). 

Ill-structured  problem-solving  situates  learning  in  authentic  environments  and  requires  the  learner  to  use 
higher  order  skills  for  reasoning  and  knowledge  application  (Ge,  Chen,  & Davis,  2005;  Jonassen,  1997).  Despite  the 
above  challenges,  learning  to  solve  technology  integration  problems  with  divergent  and  critical  thinking  can  be  a 
beneficial  way  for  preparing  preservice  teachers  to  teach  with  technology.  The  essential  responsibility  of  teacher 
educators  is  to  design  learning  environments  that  can  provide  contexts  and  scaffolds  for  ill-structured  problem- 
solving (Hannafin,  Land,  & Oliver,  1999;  Jonassen,  1997;  van  Merrienboer  & Kirschner,  2001)  so  as  to  enhance 
preservice  teachers'  understanding  and  practical  skills  of  teaching  with  technology.  According  to  literature  on 
problem-solving,  the  core  element  of  the  context  is  the  case  that  involves  contextualized  problems  that  the  learners 
must  solve  (Jonassen,  1997).  For  technology  integration,  preservice  teachers  should  be  provided  with  cases  centered 
by  instructional  problems  that  can  be  solved  with  a variety  of  technology-integrated  teaching  strategies.  Moreover, 
procedural,  elaborative,  and  metacognitive  scaffolds  should  be  designed  to  facilitate  the  ill-structured  problem 
solving  process  (Ge,  Chen,  & Davis,  2005). 

To  sum  up,  developing  lessons  incorporating  technology  into  teaching  and  learning  is  an  ill-structured 
problem-solving  process  that  requires  divergent  and  critical  thinking  skills.  As  novice  lesson  designers,  preservice 
teachers  will  encounter  challenges  in  solving  technology  integration  problems  and  may  not  use  divergent  and  critical 
thinking  skills  automatically.  Both  problem  scenarios  and  scaffolding  strategies  are  needed  to  prepare  preservice 
teachers  to  teach  with  technology  through  ill-structured  problem  solving. 

Methods 

This  study  was  conducted  on  an  introductory  technology  integration  course  at  a southeast  university  for  two 
semesters.  The  course  instructor  and  the  researcher  collaborated  on  developing  three  case  scenarios  and  the  problem 
solving  scaffolds  (See  Appendix  A for  example).  Each  of  the  cases  described  instructional  dilemmas  reflecting  a 
particular  purpose  of  teaching  with  technology,  including  promoting  communication  and  collaboration,  creativity 
and  innovation,  and  critical  thinking  skills.  The  scaffolds  outlined  the  major  steps  of  the  problem-solving  process 
and  intended  to  promote  students’  divergent  and  critical  thinking  skills.  For  each  case  project,  the  students  were 
asked  to  identify  and  solve  the  instructional  problems  by  developing  a technology-integrated  lesson  plan.  They  were 
required  to  strictly  follow  the  scaffolds  in  case  one  and  were  allowed  to  skip  some  of  the  steps  in  case  two  and  three. 

Data  Collection 

The  researcher  used  a qualitative  research  design  to  study  preservice  teachers’  experience  and  perceptions 
about  the  guided  problem  solving  approach.  To  recruit  research  participants,  the  researcher  read  the  students’  online 
autobiography  and  invited  those  with  a clear  goal  of  becoming  school  teachers  to  participate  in  this  study.  Four 
students  - two  from  the  first  semester  and  two  from  the  second  semester  - volunteered  to  participate  in  a one-hour 
semi-structured  interview.  The  interviews  were  conducted  and  audio  recorded  at  the  end  of  each  semester.  The 
interview  questions  began  with  the  participants’  previous  experience  with  teaching  and  learning,  and  then  focused 
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on  their  scaffolded  instructional  problem-solving  process  as  well  as  their  perceptions  about  the  problem-solving 
activities.  For  this  paper,  the  researcher  selected  to  present  an  analysis  two  interviews  from  each  semester.  The 
participants’  responses  to  scaffolds  and  their  lesson  plans  were  also  collected  for  analysis. 

Participant  description 

The  four  participants  came  from  three  different  teacher  preparing  programs.  Jason  was  a junior  student  in 
technology  education  major.  He  had  participated  in  one  practicum,  during  which  he  developed  and  implemented  a 
lesson  on  engineering  design  process.  Claire  was  a sophomore  student  in  early  childhood  education  major  with  a 
focus  on  math  and  science.  She  had  been  teaching  crafts  class  to  young  students  in  the  enrichment  program  of  a 
local  elementary  school.  Brian  was  a sophomore  student  in  middle  school  education  major  with  a focus  on  math  and 
science.  He  had  not  worked  with  middle  school  students  he  wanted  to  teach  but  he  had  been  teaching  bible  studies  to 
youths  at  the  church.  Sarah  was  also  a sophomore  student  in  middle  school  education  major,  concentrating  on 
language  and  math.  Along  with  the  technology  integration  course,  she  was  taking  another  course  requiring  48  hours’ 
practicum  in  a seventh  grade  math  class  and  a seventh  grade  language  arts  class. 

Data  Analysis 

The  researcher  used  grounded  theory  methods  (Glaser  & Strauss,  1967)  to  analyze  the  data.  Strauss  and 
Corbin  (1990)  believe  grounded  theory  is  “discovered,  developed,  and  provisionally  verified  through  systematic  data 
collection  and  analysis  of  data  pertaining  to  that  phenomenon’’.  The  researcher  conducted  systematic  analysis  by 
using  techniques  such  as  open  coding,  focused  coding,  and  theoretical  categorizing.  She  tried  to  identify  inductive 
codes  and  categories  in  analyzing  the  first  two  interviews  and  applied  the  typical  categories  to  the  other  interviews 
as  a way  of  focused  coding.  Constant  comparison  was  used  in  both  creating  and  sorting  codes  to  generate  categories. 
Ezzy  (2002)  describes  constant  comparison  as  a process  of  developing  and  identifying  codes  that  can  be  compared 
for  similarities  and  differences  and  believes  “comparisons  allow  data  to  be  grouped  and  differentiated,  as  categories 
are  identified  and  various  pieces  of  data  are  grouped  together”  (p.  90).  With  tables  of  categories  and  quotations  from 
all  four  interviews,  the  researcher  searched  for  common  themes  across  the  participants  as  well  as  the  strong  themes 
within  each  individual.  As  themes  were  identified,  the  researcher  wrote  memos  to  describe  them  and  attached  the 
quotations  as  evidence  for  supporting  the  themes.  As  Ezzy  (2002)  argues,  “sophisticated  use  of  grounded  theory 
draws  on  both  inductive  and  deductive  methods  of  theory  generation  (p.  12)”.  For  the  last  step  of  data  analysis, 
relationships  among  the  inductively  generated  themes  and  categories  were  analyzed  and  connected  to  the  existing 
knowledge  in  literature. 


Findings 

Based  on  coding  the  transcriptions  and  generating  categories,  six  major  themes  anchored  around  the  three 
research  questions  emerged  from  the  data,  such  as  the  scaffolds  promoted  divergent  and  critical  thinking  to  some 
extent;  the  preservice  teachers  used  self-experience  and  made  inference  in  identifying  and  solving  the  problems; 
although  the  preservice  teachers  developed  new  perceptions  about  problem  solving,  their  underlying  assumptions 
remained  unchallenged.  These  themes  were  identified  either  because  they  were  repeated  by  more  than  one 
participant  or  because  they  represent  a strong  idea  throughout  one  participant's  entire  interview.  Each  major  theme 
involved  multiple  dimensions  or  sub  themes  that  were  closely  related  to  one  another.  The  themes  will  be  presented 
following  the  research  questions  of  this  study. 

Roles  of  the  Scaffolds  in  Facilitating  Problem  Solving 
Promoting  Divergent  Thinking  to  Some  Extent 

Encouraged  by  the  problem-solving  scaffolds  to  create  as  many  ideas  as  possible,  the  preservice  teachers 
were  able  to  use  some  divergent  thinking  skills  in  finding  multiple  problems  and  generating  alternative  solutions. 
However,  their  divergent  thinking  was  hindered  by  the  lack  of  real  world  experience  as  a teacher  and  their  focus  on 
previous  experience  as  a student.  They  also  had  inadequate  understanding  of  the  purpose  to  use  divergent  thinking. 

Divergent  problem-finding  promoted  the  preservice  teachers  to  identify  the  most  essential  problem  as  well  as 
to  consider  the  other  possible  problems  as  constraints  to  be  addressed.  Sarah  said,  “I  wrote  down  all  of  the  problems 
that  I could  find,  just  everything  I could  think  of’  and  then  “picked  the  one  that  I thought  was  really  the  biggest 
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challenge  that  would  help  the  biggest  number  of  students.”  Claire  mentioned  the  problem  that  she  would  overlook 
without  brainstorming,  “Another  detail  I would  have  skipped  over  like  in  the  challenges  was  you  know  there  are 
some  take  charge  students.”  She  realized  that  she  had  to  “monitor  the  groups  to  make  sure  those  students  do  not 
dominate  the  other  students”.  In  her  final  lesson  plan,  she  decided  to  make  groups  “according  to  students’  ability  and 
work  ethic”. 

Divergent  thinking  was  evident  in  generating  solutions  of  teaching  with  technology,  especially  in  the  first 
case  project  when  they  followed  the  scaffolds  strictly.  Three  preservice  teachers  considered  a variety  of  technology 
tools  that  could  be  integrated.  For  example,  with  the  problem  that  students  may  have  difficulty  accessing 
information  for  the  project  of  identifying  career  opportunities,  Jason  said  he 

added  in  a Delicious  Account  that  would  make  the  student  bookmark  websites  that  would  be  related  to  the 
career  they  chose. . . required  each  group  to  access  a set  number  of  Blogs  related  to  the  career  pathway  they 
choose. . . I used  the  Gmail  accounts  as  well  within  the  groups  to  promote  communication  and  collaboration 
within  the  groups. 

Along  with  brainstorming,  Jason  and  Claire  took  an  open  attitude  toward  the  divergent  solutions  they 
generate.  Jason  said.  “Whether  I thought  they  would  be  used  or  not,  I wanted  to  right  down  and  just  see  if  they 
would  be  used.”  Claire  also  mentioned,  “I  put  down  like  I was  going  to  do  a Photostory. . . I put  this  down  as  a 
solution  but  I don't  think  that’s  really  going  to  be  what  I wanted  to  do.”  They  both  reported  that  they  ended  up  using 
a better  solution  different  from  the  one  they  preferred  at  the  beginning. 

However,  with  insufficient  field  experience,  the  preservice  teachers  had  difficulty  identifying  a variety  of 
“real”  challenges  they  might  encounter  in  a classroom  setting.  Jason  felt  “it  was  actually  tougher  than  I thought 
would  be. . . I had  to  think  of  something  to  say  they  are  easily  getting  off  task,  they  are  not  paying  attention  to  what 
you  do”.  Brian  found  his  only  difficulty  was  to  “get  enough  challenges”,  and  he  actually  “started  making  up 
different  challenges”.  They  were  imaging  the  potential  problems  based  on  prior  experience.  More  importantly,  they 
tended  to  use  convergent  thinking  when  relying  on  critical  personal  experience  relevant  to  the  problem  situation. 

The  scaffolds’  role  in  supporting  divergent  thinking  was  very  limited  in  such  cases.  For  example,  in  resolving  the 
second  case  on  communication  and  collaboration,  Brian  referred  to  his  experience  as  a higher-achieving  student  and 
believed  that  the  problem  in  case  two  was  “trying  to  teach  the  higher-achieving  students  and  not  hold  them  back, 
while  also  teach  the  lower-achieving  students”.  His  solution  was  to  “group  them  up  and  have  them  work  together 
and  in  hopes  that  the  lower-achieving  students  will  learn  from  the  higher  achieving  students”.  With  confidence  in  his 
ideas,  Brian  ignored  the  other  problems  portrayed  in  the  case  scenario  and  did  not  consider  alternative  solutions. 

Moreover,  the  preservice  teachers  did  not  fully  understand  the  purpose  of  brainstorming  and  regarded  it  as  a 
course  requirement,  which  was  another  factor  inhibiting  divergent  thinking.  Both  Claire  and  Sarah  valued  quality 
over  quantity  when  working  on  the  second  and  third  case.  For  example,  Claire  said  in  case  two,  “I  came  up  with  two, 
kind  of  pretty  in-depth  ones  to  choose  from.  I could  have  expanded  them  into  three  or  four  but  I tried  to  really 
narrow  it  into  two  that  I thought  would  be  most  applicable.”  Jason  also  restrained  from  divergent  thinking  because 
of  his  initial  preference  and  time  limitations. 

Although  the  preservice  teachers  reported  brainstorming  ideas  in  finding  problems  and  generating  solutions, 
particularly  in  the  first  case,  these  actions  were  not  sustainable  throughout  the  three  case  projects.  They  regarded 
divergent  thinking  as  a requirement  in  the  scaffolds  and  chose  to  skip  over  when  the  scaffolds  became  optional. 

They  did  not  realize  that  the  purpose  of  thinking  divergently  was  to  explore  even  better  ideas.  Also,  as  mentioned 
earlier,  excessive  reliance  on  personal  experience  as  a student  and  the  lack  of  real  world  teaching  experience  limited 
the  preservice  teachers’  ability  to  think  of  divergent  ideas. 

Facilitating  Critical  Thinking  in  Problem  Solving 

The  scaffolding  steps  of  identifying  challenges,  framing  problems,  and  creating  criteria  to  evaluate  the 
solutions  facilitated  the  preservice  teachers’  critical  thinking.  However,  when  the  preservice  teachers  focused  their 
attention  on  generating  solutions  only,  they  were  less  likely  to  use  critical  thinking  skills. 

The  preservice  teachers  had  an  initial  tendency  to  overlook  the  complexity  of  problem  analysis  and  shift  to 
solution  generation  rapidly.  Brian  believed  that  “it  was  pretty  evident  [what]  the  instructional  problem  was  in  the 
case,  so  the  real  big  problem  is  solving  it.”  So  he  mainly  focused  on  “the  way  I was  going  to  teach  it”.  Claire  also 
tended  to  focus  on  solutions  before  truly  identifying  the  problems.  She  said  that  after  reading  the  first  case  scenario 
“I  had  some  ideas. . . automatically  in  my  head.” 
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However,  by  following  the  scaffolds  to  articulate  the  challenges  and  problems  before  generating  solutions, 
three  of  the  preservice  teachers  realized  the  need  to  clarify  the  problem  space.  As  Claire  was  working  on  her  second 
case  project  of  promoting  students’  creativity  in  learning  American  Revolution,  she  paid  more  attention  to  her 
problems  and  goals.  After  finishing  all  the  cases,  Claire  believed  that 

I can't  just  say  okay  this  is  what  I want  to  do  and  just  go  and  do  it.  I have  to  look  at  each  of  these  different 
things  I am  going  to  encounter,  like  the  different  standards,  the  different  challenges,  and  think  about  that 
and  figure  out  what  would  be  best  in  the  end. 

Sarah  also  found  it  worthwhile  to  spend  time  thinking  critically  at  the  beginning  stages  of  problem-solving.  She  said 
the  scaffolds  gave  her  “a  better  way  to  make  sure  that  I am  using  the  right  tool  to  solve  the  problem”. 

Metacognition  enables  the  problem  solver  to  actively  monitor  and  control  his  or  her  problem-solving 
performance  based  on  goals.  Novice  problem  solvers  usually  have  limited  metacognitive  abilities.  However,  while 
using  the  scaffolds,  the  preservice  teachers  demonstrated  metacognitive  thinking  skills. 

Jason  mentioned,  in  selecting  his  criteria,  he  “thought  back  to  what  problems  were  most  prevalent”  and  tried 
his  best  to  “make  sure  the  criteria  fit  the  lesson  [and]  the  instructional  problem”.  Both  Claire  and  Sarah  looked  at  the 
curriculum  standards  they  wanted  to  teach,  the  characteristics  of  their  students,  and  their  pedagogical  goals  such  as 
“create  enough  jobs  for  everyone  involved”,  “keep  the  students  on  task”,  “encourage  critical  thinking  and  problem 
solving”,  etc. 

Moreover,  Jason  and  Claire  talked  about  using  the  criteria  to  monitor  and  adjust  their  lesson  plans. 
Considering  the  criteria  that  the  solution  should  help  “develop  appropriate  limitations  and  requirements”  for  students, 
Jason  refined  his  solution  in  designing  the  lesson  plan. 

I had  to  go  back  and  think  well  how  could  this  rubric  help  me  with  limitations?  And  I had  to  reorganize  my 
thoughts  on. . . they  [students]  are  not  just  going  to  develop  a rubric,  they  are  going  to  develop  a rubric  that 
they  believe  they  can  accomplish. . . they  set  the  task  that  they  are  willing  to  try. 

The  scaffolding  step  of  using  criteria  to  grade  the  potential  solutions  assisted  the  preservice  teachers’ 
decision  making  process.  Three  of  the  preservice  teachers  critically  evaluated  their  potential  solutions  and  realized 
the  benefits  of  making  decisions  based  on  criteria.  Before  the  evaluation  in  case  one,  Jason  thought  “another  one 
[solution]  would  win”,  but  as  he  applied  the  criteria  to  the  solutions,  he  found  “that’s  [that  solution  was]  not  as  good 
as  I thought  was  going  to  be”  and  finally  chose  a different  solution  that  was  “a  better  fit”.  Claire  described  similar 
experience  in  case  one, 

On  my  first  case,  I was  like  “Oh  yeah  this  is  the  best  solution  for  this  project”.  But  when  I really  thought 
about  is  this  really  going  to  meet  all  of  these  criteria  that  I think  needs  to  be  met,  it  ended  up  being  a 
different  one. 

Nevertheless,  there  were  cases  when  the  preservice  teacher  failed  to  use  critical  thinking  while  following  the 
scaffolds  ostensibly.  As  mentioned  earlier,  Brian  focused  more  on  generating  solutions  to  the  problem  he  found.  His 
exclusive  emphasis  on  the  solutions  also  blinded  him  of  the  complexity  in  creating  criteria  and  evaluating  solutions. 
Brian  found  it  “pretty  simple”  to  develop  criteria,  which  were  based  on  the  solutions  he  preferred  rather  than  his 
goals  of  solving  the  instructional  problems. 

1 decided  it  [criteria]  rather  quickly.  After  I figured  out  I need  to  use  this  kind  of  learning  and  this  kind  of 
learning.  Then  I could  make  my  criteria  because  I could  base  it  on  how  I am  going  to  teach  my  kids. 

As  a result,  Brian  found  all  the  solutions  were  satisfying  and  realized  that  “I  could  use  all  of  these”.  For  example,  in 
case  one,  instead  of  judging  the  strength  of  the  solutions,  he  finally  combined  all  of  them  into  a single  one,  believing 
that  it  would  “better  aid  my  students”  and  could  be  used  in  “pretty  much  any  situation”.  Brian’s  lack  of  critical 
thinking  reflected  his  limited  understanding  of  the  purpose  for  making  criteria  and  evaluating  the  solutions. 

In  general,  with  the  assistance  of  the  scaffolds,  the  preservice  teachers  were  able  to  thinking  critically  by 
clarifying  the  problem  space  before  generating  solution,  using  metacognitive  skills  to  monitor  and  control  the 
problem-solving  process,  and  judging  the  strength  of  potential  solutions  based  on  criteria.  However,  when  the 
preservice  teachers  failed  to  recognize  the  complexity  behind  finding  problems  and  selecting  solutions,  they  were 
less  likely  to  be  aware  of  the  need  for  critical  thinking. 

Presen’ice  Teachers’  Strategies  for  Solving  Problems 
Using  Self  Experience  to  Identify  Problems  and  Solutions 

Because  the  case  scenarios  were  situated  in  a generic  classroom  setting,  without  specifying  grade  levels  and 
subject  areas,  the  preservice  teachers  had  to  adapt  them  to  their  own  context  while  working  on  the  problems.  They 
frequently  used  their  previous  experience,  mostly  as  a student  and  sometimes  as  a teacher,  to  help  problem  solving. 
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Three  of  the  preservice  teachers  mainly  relied  on  their  experience  as  a student  in  K-12  school  or  college  to 
find  the  problems  they  might  encounter.  When  Claire  was  talking  about  her  potential  challenges  in  teaching  the 
standard  on  making  graphs  with  data,  she  said,  “I  can  remember  when  I was  doing  graphs. . . that  a lot  of  students 
have  Uouble  with  it.  And  I was  like  is  this  an  area  that  fourth  graders  struggle  with.”  In  the  case  of  using  technology 
to  promote  communication  and  collaboration,  Sarah  also  referred  to  the  problem  she  experienced  in  her  Freshmen 
English  Writing  class. 

We  did  the  peer  review  but  we  didn't  do  it  online  we  just  brought  our  paper  in  class.  And  I thought  about 
how  it  got  so  messy  and  it  was  so  confusing  for  me  to  see.  Are  they  talking  about  this  in  it  or  are  they 
talking  about  that  in  it,  so  then  I used  my  own  problem  that  I had  to  figure  out  a better  solution  than  the  one 
I had  experienced. 

The  problems  identified  by  the  preservice  teachers  based  on  personal  learning  experience  were  related  to  but 
different  from  those  described  in  the  case  scenarios,  such  as  students'  unequal  participation,  lack  of  mutual 
understanding,  and  diversified  learning  abilities.  Previous  experience  focused  the  preservice  teachers’  attention  on 
the  problems  they  were  familiar  with  as  a student,  but  were  unable  to  support  their  exploration  of  the  other  problems 
that  should  also  be  addressed  by  the  teacher. 

Different  from  the  others,  Jason  related  to  his  practicum  experience  when  looking  for  instructional  problems. 
As  he  reported, 

In  all  three  of  the  cases,  I immediately  referred  back  to  the  practicum  I did  earlier. . . I found  that  a lot  of 
students  usually  lose  focus  quickly  if  they  don't  have  enough  things  to  do.  Students  didn’t  have  or  unable  to 
access  stuff  on  their  own. 

Jason  identified  problems  not  only  from  his  own  observation  but  also  from  his  communication  with  the  mentor 
teacher,  who  mentioned  the  situation  that  students  did  not  want  to  explore  questions  on  their  own  but  expected  the 
teacher  to  give  the  answer.  Jason  used  this  problem  in  his  third  case  project  on  critical  thinking.  Although  the 
instructional  problems  Jason  found  were  not  derived  from  the  case  scenarios  either,  they  reflected  a teacher’s 
perspective  in  contrast  with  the  students’  perspective  adopted  by  the  other  preservice  teachers. 

In  terms  of  generating  solutions,  previous  learning  experience  played  a major  role  as  well.  However,  since 
the  preservice  teachers  were  not  taught  with  technology  when  they  were  in  K-12  classroom,  they  tried  to  incorporate 
technology  into  the  learning  activities  they  had  experienced.  For  instance,  Sarah  adapted  the  peer  review  activity  in 
her  Freshmen  English  Writing  class  to  her  seventh  grade  language  arts  class.  She  designed  the  solution  of  having 
students  write  a paper  in  Google  Docs  and  review  partners’  writing  by  “highlighting  and  adding  comments”. 

Because  most  of  the  preservice  teachers  attending  the  course  were  at  the  preliminary  stage  of  their  teacher 
preparation  program  and  had  not  taken  any  method  courses  yet,  they  lacked  pedagogical  content  knowledge,  which 
was  an  integral  component  of  successful  technology  integration.  As  a result,  empirical  experience  with  different 
learning  activities  became  the  major  source  for  them  to  compensate  pedagogical  content  knowledge. 

Nevertheless,  both  Jason  and  Sarah,  who  had  been  observing  real  classrooms  of  their  subject  area,  used  their 
field  experience  to  evaluate  the  appropriateness  of  the  learning  activities  for  their  targeted  students.  Jason  said,  in 
making  his  lesson  plans,  he  “thought  of  previous  experience  with  students. . . what  ways  did  work  and  what  ways 
didn't  work.”  Sarah  also  emphasized  her  strength  of  using  observation  experience  for  making  good  judgments. 

I can  kind  of  think  OK  I have  seen  my  teacher  do  this  with  seventh  graders  and  they  love  it,  or  I have  seen 
my  teacher  trying  to  do  something  like  this  and  they  don't  even  want  to  try.  So  just  by  being  in  the  seventh 
grade  classroom  that  semester. . . I think  I had  a better  idea  about  if  the  kids  would  actually  want  to  do  it  and 
if  it  would  actually  work  in  a classroom. 

Overall,  although  the  preservice  teachers  can  use  their  own  learning  experience  for  identifying  instructional 
problems  and  designing  solutions,  the  lack  of  teaching  experience  was  still  a big  barrier  for  them  to  perform  in  a 
professional  way.  The  preservice  teachers  with  relevant  real  world  experience  showed  some  advantages,  as  they 
were  better  able  to  find  real  problems  and  judge  the  feasibility  of  their  solutions. 

Making  Inferences  to  Generate  and  Evaluate  Solutions 

When  the  preservice  teachers  had  little  direct  experience,  either  as  a student  or  a teacher,  to  rely  on,  they 
tended  to  make  inferences  with  the  information  they  identified  in  the  problem  space.  This  strategy  was  most  evident 
as  the  preservice  teachers  were  generating  ideas  of  teaching  with  technology  or  anticipating  the  effectiveness  of  their 
solutions. 
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As  mentioned  earlier,  the  preservice  teachers  had  not  used  the  latest  technology  tools  when  they  were  K-12 
students,  making  it  impossible  for  them  to  directly  use  the  same  learning  activities  from  their  classrooms.  Based  on 
analyzing  the  process  that  the  preservice  teachers  generated  technology  integration  solutions,  two  major  ways  of 
making  inferences  emerged:  activity-based  inference  and  tool-based  inference. 

While  using  activity-based  inference,  the  preservice  teachers  first  designed  a learning  activity,  which  usually 
originated  from  their  previous  experience,  to  address  their  problems  or  goals  and  then  added  some  tool(s)  to  enhance 
that  activity  with  technology’s  affordances.  Take  Brian  for  example,  he  designed  the  group  activity  involving  both 
higher-achieving  and  lower-achieving  students  to  solve  his  problem  and  incorporated  Smart  Board,  arguing  that  “it’s 
not  like  write  on  chalk  board  or  write  and  erase  board,  they  can  do  it  on  Smart  Board  and  then  1 can  go  over  and 
correct  it  and  other  students  can  correct  it”.  Activity-based  inference  helped  the  preservice  teachers  concentrate  on 
the  problems  and  goals  of  their  teaching  situation.  However,  technology  was  likely  to  be  something  dispensable  if 
the  match  between  the  pedagogical  activity  and  the  technology’s  affordances  was  inappropriate. 

While  using  tool-based  inference,  the  preservice  teachers  first  picked  a technology  tool  with  functions 
pertinent  to  the  content  they  wanted  to  teach  and  then  designed  learning  activities  based  on  that  connection  to 
address  their  instructional  problems.  Claire  thought  about  using  Web  2.0  tool  to  facilitate  communication  and 
collaboration  in  learning  to  make  graph.  She  said  she  “researched  some  different  Web  2.0  tools  to  see  if  there  is 
something  for  how  to  make  a graph  or  ways  to  find  data”  and  chose  to  use  Delicious  Account  for  students  to  “find 
and  share  research  [where  they  can  gather  data]  they  like  with  their  fellow  students”  and  use  Mind  Meister,  a web 
concept  mapping  tool,  for  students  to  “add  information  to  a collaborative  mind  map  to  organize  the  information”. 
Tool-based  inference  helped  the  preservice  teachers  identify  good  connections  between  technology’s  affordances 
and  the  content  and  ways  of  learning.  However,  this  strategy  tended  to  focus  the  preservice  teachers’  attention  on  the 
tool  they  prefer  to  use  rather  than  driving  them  to  explore  a variety  of  choices. 

Another  way  of  making  inferences  happened  when  the  preservice  teachers  were  envisioning  the  results  of 
their  solution  ideas  in  order  to  make  decisions.  Due  to  the  lack  of  field  experience,  the  preservice  teachers  did  not 
have  sufficient  experiential  knowledge  to  judge  the  effectiveness  of  their  solutions.  Also  because  of  the  limitation  of 
the  course,  they  had  no  chance  to  test  their  ideas.  Therefore,  two  preservice  teachers  made  inferences  through  role 
playing  or  imagination  to  predict  the  results  of  implementing  the  learning  activity  they  designed.  Jason  positioned 
himself  as  a student  when  reasoning  about  the  benefits  of  asking  students  to  research  their  preferred  careers  online 
and  share  the  resources  using  Delicious  Account. 

I actually  thought  back  and  said  well  if  I was  forced  to  use  this  as  the  teacher  assigned  this  to  me,  would  I 
have  done  any  better?  And  I believe  I would  have.  I believe  the  outcome  would  have  been  different.  I 
believe  I would  know  more...  So  I believe  the  students  would  be  interested  in  doing  this  kind  of  work. 
Similarly,  Claire  gave  up  her  initial  idea  of  asking  students  to  make  graphs  on  “band  ticket  sales”  in  case  one  after 
imagining  the  possible  consequences. 

It  was  beneficial  for  preservice  teachers  to  look  ahead  by  reasoning  through  the  appropriateness  of  their 
learning  activity  design.  However,  because  of  limited  experience-based  evidence,  all  the  three  types  of  inferences  - 
activity-based  inference,  tool-based  inference,  and  imagination-based  inferences  - may  be  inaccurate  thus 
misleading  their  judgments. 

Preservice  Teachers  ’ Assumptions  and  Perceptions  about  Problem  Solving 
Assumptions  Underlying  the  Problem  Solving  Process 

The  preservice  teachers  had  assumptions  and  beliefs  about  themselves  and  their  students,  which  they  took  for 
granted  throughout  the  problem  solving  and  lesson  designing  process.  Originated  from  their  prior  experience  of 
learning  and  teaching,  these  assumptions  and  beliefs  had  become  their  rationale  for  using  certain  strategies  to  design 
lesson  activities. 

The  preservice  teachers  also  had  assumptions  about  their  content  and  pedagogical  knowledge.  Two  of  the 
preservice  teachers  believed  that  they  had  sufficient  understanding  about  the  teaching  content  or  the  pedagogy  to  be 
used,  so  that  they  did  not  need  to  consult  additional  resources.  In  designing  the  lesson  on  American  Revolution, 
Claire  reported  that 

I felt  a lot  of  it  [information  about  revolutionary  war]  came  from  my  mind  because  I am  a huge  history  buff 
too.  So  ((laughing))  I could  think  of  the  different  events  of  the  war,  without  necessarily  having  to  go  and 
look  at  them  from  different  sources. 

Brian  was  very  confident  in  his  knowledge  about  teaching  methods  as  well  as  his  solution  for  approaching  the 
situations  he  faced. 
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Actually  a lot  of  methods  just  came  out  of  my  head,  because  of  my  past  three  educational  classes.  I just 
kind  of  knew  that  you  need  to  have  hands-on  learning,  just  different  stuff  like  that.  I just  kind  of  already 
knew  that  in  my  head  a little  bit.  So  I didn't  have  to  use  the  Internet  as  much  to  find  ways  to  teach. 

Although  these  assumptions  suggested  that  the  preservice  teachers  had  some  preparation  on  content  and  pedagogy, 
which  was  the  prerequisite  for  solving  the  instructional  problems,  they  blocked  their  metacognitive  awareness  of  the 
need  to  further  explore  the  teaching  content  and  strategies  and  how  to  integrate  them  with  technology. 

Moreover,  the  preservice  teachers  had  common  assumptions  about  the  students’  problem  with  learning,  their 
preferred  activities,  and  technology's  role  in  learning.  They  almost  reached  an  agreement  on  the  major  instructional 
problem  they  would  encounter  in  their  class.  That  was,  the  students  were  unable  to  concentrate  on  study  and  would 
easily  get  off-task.  Jason  found  that  students  “usually  lose  focus  quickly  if  they  don't  have  enough  things  to  do”. 
Claire  believed  that  “students  were  so  easily  distracted  that  they  move  off  task”.  Brian  also  mentioned  “having 
trouble  keeping  the  kids'  attention”.  Their  assumptions  about  how  this  problem  could  be  solved  were  also  quite 
similar:  the  students  needed  interesting  and  entertaining  activities  to  keep  their  focus  or  attention.  Jason  believed  that 
“the  work  often  times  has  to  be  new  to  them,  something  different  and  something  interesting”.  Brian  thought  “when 
they  [students]  are  moving  around  a little  more,  going  outside,  coming  back  in,  you  could  actually  teach  more”.  The 
preservice  teachers  believed  it  important  to  integrate  technology  mainly  for  two  reasons:  students  were  comfortable 
with  technology  and  would  be  more  attentive  in  learning  with  technology.  Claire  said,  “Since  this  is  a digital  age, 
students  are  growing  up  in  that,  and  they  will  really  need  some  of  that  incorporated  into  their  learning.”  Brian 
assumed  that  “technology  solves  the  problem  getting  the  curriculum  across  to  the  students”  because  students  were 
“technologically  sound”.  Both  Jason  and  Brian  believed  that  hands-on  activities  with  technology  would  keep 
students  “on-task  and  focused”  because  “they  [students]  can  create  things  that  they  wouldn't  be  able  to  create 
otherwise”  and  “they  [students]  can  see  it  with  their  eyes  and  learn  it  in  that  [hands-on]  way”. 

With  all  these  assumptions  about  their  knowledge  and  the  learner  characteristics,  the  preservice  teachers 
were  very  likely  to  design  a “fun”  lesson  that  would  work  for  them  as  a student  to  keep  their  envisioned  students  on- 
task,  especially  when  they  failed  to  consider  multiple  instructional  problems  and  did  not  use  proper  criteria  to 
evaluate  their  solutions. 

Newly  Developed  Perceptions  on  Solving  Instructional  Problems 

Although  the  preservice  teachers’  assumptions  about  themselves  and  their  students  were  not  challenged,  their 
perceptions  about  the  instructional  problem  solving  process  evolved  as  they  were  using  the  scaffolds,  which 
positively  influenced  their  understandings  regarding  the  ways  of  designing  a better  lesson. 

The  preservice  teachers  realized  the  importance  of  clarifying  the  problems  and  goals  before  setting  out  to 
solve  the  problems.  Jason  found  it  necessary  to  “not  only  develop  a lesson  but  go  through  this  process  to  make  sure 
my  students  are  getting  the  most  out  of  a lesson.”  Claire  also  emphasized  that  “I  have  to  look  at  each  of  these 
different  things  I am  going  to  encounter,  like  the  different  standards,  the  different  challenges,  and  think  about  that 
and  figure  out  what  would  be  best  in  the  end".  Both  of  them  were  conscious  of  keeping  their  goals  and  problems  in 
mind  to  adjust  their  solutions.  Claire  and  Sarah  reflected  on  their  tendency  of  sticking  to  the  initial  idea.  Claire  said. 

“I  can’t  go  straight  to  the  solution  any  more  I have  to  actually  go  through  and  think  about  the  details  a little.”  Sarah 
found  that  “it  took  more  time  at  the  beginning  to  do  it  all,  but  every  project  turned  to  work  exactly  how  I wanted  it  to 
work.  So  just  go  through  all  the  steps  and  not  just  pick  something  because  it  sounded  really  good  to  begin  with.” 
Their  emphasis  on  analyzing  the  situation  before  making  a decision  reflected  their  increased  awareness  of 
considering  alternative  choices  and  environmental  constraints  in  solving  problems.  Additionally,  both  Jason  and 
Brian  indicated  their  understanding  about  the  significant  efforts  needed  in  lesson  design.  Jason  said,  “I  should  think 
a lot  harder  about  my  lessons  when  I design  them”.  Brian  said  he  learned  that  “when  I am  a teacher  it’s  [the  lesson  is] 
not  just  going  to  unfold  for  me.  I have  to  go  and  research  and  find  out  stuff  by  myself’.  As  novice  lesson  designers, 
they  began  to  have  a sense  of  the  complexity  involved  in  lesson  planning. 

The  preservice  teachers  also  developed  some  new  perceptions  about  divergent  and  critical  thinking.  They 
talked  about  the  benefits  of  thinking  divergently  and  implied  the  need  to  be  open-minded  toward  the  initial  ideas. 
Jason  said.  “Before  case  one  I would  say  there’s  probably  not  much  need  to  do  a lot  of  brainstorming.”  But  after 
working  on  that  case,  he  realized  “if  I hadn't  done  the  brainstorming  I would  have  chose  a solution  that  probably 
wasn’t  the  best  one.”  Therefore,  he  believed  that  thinking  of  “many  possible  solutions”  rather  than  sticking  to  a 
convergent  solution,  would  benefit  the  teacher  and  the  students.  In  terms  of  critical  thinking,  the  preservice  teachers 
pointed  out  the  value  of  creating  and  applying  criteria.  Brain  believed  that  “it’s  important  for  us  to  realize  what  kind 
of  criteria  we  had  to  meet  to  effectively  teach  them  [the  students]  in  the  classroom”.  Claire  stressed  that  creating 
criteria  helped  her  “really  concentrate  and  figure  out  what  is  the  most  important  overall”.  The  preservice  teachers 
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also  perceived  it  as  an  effective  way  to  distinguish  the  better  solutions  based  on  criteria.  Sarah  said,  “It  really  helped 
to  be  able  to  see  on  a scale  of  one  to  ten,  does  this  work  with  what  my  criteria  are.”  In  the  second  case,  when  she 
was  not  required  to  follow  the  scaffolds,  she  still  evaluated  the  solutions  using  her  two  main  criteria  - being  able  to 
solve  the  problem  and  using  technology  integral  to  the  project  - indicating  her  awareness  of  thinking  critically  about 
the  potential  solutions. 

By  following  the  scaffolding  procedure,  the  preservice  teachers  not  only  improved  their  performance  but  also 
developed  new  perspectives  and  attitudes  toward  instructional  problem  solving,  which  was  evidenced  by  their 
understanding  of  identifying  details  in  the  problem  space,  paying  efforts  to  lesson  design,  thinking  divergently  about 
initial  ideas,  and  making  decisions  based  on  criteria.  However,  these  new  understandings  were  focusing  on  the 
instructional  problem  solving  skills,  independent  from  their  assumptions  about  themselves  and  their  students.  The 
preservice  teachers'  pedagogical  content  knowledge  may  not  develop  significantly  along  with  the  new  perceptions. 

Discussion 

Data  analysis  revealed  that  both  the  problem-solving  scaffolds  and  the  preservice  teachers’  own  strategies 
were  influencing  the  way  they  addressed  the  instructional  problems  with  technology  integrated  lesson  plans. 

Analysis  of  preservice  teachers’  perceptions  about  their  problem  solving  experience  indicated  two  categories 
respectively  associated  with  the  scaffolds  and  the  strategies.  Figure  1 summarized  the  major  findings  and  their 
connections. 

Figure  1.  Major  Findings  of  the  Study 
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The  preservice  teachers  were  able  to  use  divergent  and  critical  thinking  skills  during  problem  solving.  Some 
of  them  carefully  explored  multiple  problems  in  their  teaching  situation,  and  all  of  them  considered  different 
technology  tools  in  generating  potential  solutions.  They  also  began  to  think  critically  by  clarifying  the  details  in  the 
problem  space  as  well  as  monitoring  and  controlling  the  problem-solving  process  with  goals  and  criteria.  Compared 
with  the  preservice  teachers’  initial  tendency  of  abiding  by  convergent  solutions,  their  problem-solving  performance 
had  been  enhanced  by  following  the  scaffolding  procedure.  As  the  preservice  teachers  realized  the  benefits  of 
thinking  divergently  and  critically,  they  developed  some  new  perceptions  about  designing  lessons  for  approaching 
instructional  problems.  But  due  to  the  inadequate  understanding  of  the  purpose  of  divergent  and  critical  thinking, 
although  the  preservice  teachers  talked  about  the  advantages  of  using  those  skills,  they  did  not  use  them  as  intended 
by  the  scaffolds,  especially  when  they  were  not  required  to  follow  the  scaffolds  strictly. 

Meanwhile,  the  preservice  teachers  frequently  used  their  own  experience  as  a student  or  as  a teacher  to  assist 
problem  solving.  They  also  used  activity-based  and  tool-based  inferences  for  generating  solutions  and  tried  to 
predict  the  potential  results  of  those  solutions  as  a way  of  evaluation.  These  strategies  made  it  possible  for  the 
preservice  teachers  to  complete  the  instructional  problem-solving  steps,  even  though  they  had  neither  taken  any 
method  courses  nor  started  their  field  teaching.  However,  to  some  extent,  these  strategies  also  conflicted  with  the 
scaffolds  and  negatively  influenced  the  preservice  teachers'  divergent  and  critical  thinking  performance.  Some  of 
the  preservice  teachers  failed  to  consider  alternative  problems  and  solutions  in  the  case  scenario  when  they  were 
exclusively  relying  on  their  own  experience  with  similar  teaching  situations,  in  which  the  problem  and  the  solution 
were  evident  to  them.  As  they  used  convergent  thinking,  they  were  inclined  to  underestimate  the  complexity 
involved  and  think  less  critically  about  the  solutions.  The  negative  aspect  of  the  strategies  can  be  traced  back  to 
preservice  teachers'  underlying  assumptions.  They  did  not  view  themselves  as  a teacher  or  a student  all  the  time  but 
switched  perspectives  on  their  roles  constantly.  With  the  role  of  a teacher,  they  tended  to  assume  that  they  already 
knew  enough  about  the  teaching  situation,  overlooking  the  need  to  further  inquire  into  the  problems  and  solutions  by 
using  resources  other  than  their  mind.  With  the  role  of  a student,  they  tended  to  assume  the  equivalence  between 
themselves  and  their  students  and  to  design  an  optimized  version  of  the  learning  activities  they  had  experienced  for 
their  students. 


Implications 


Preservice  Teachers  ’ Technology  Integration 

Ill-structured  design  problems  such  as  technology  integration  require  knowledge  from  multiple  domains  and 
extensive  experience  in  the  field.  The  preservice  teachers  with  deficiencies  in  both  areas  used  personal  history  based 
theories  and  experience,  which  were  tenacious  and  powerful  (Knowles  & Holt-Reynolds,  1991),  to  help  find  and 
solve  the  design  problems.  They  adapted  the  case  scenario  by  recreating  it  exclusively  based  on  what  they 
personally  witnessed  or  experienced.  Similar  to  Knowles  and  Holt-Reynolds’ s (1991)  findings,  the  preservice 
teachers  failed  to  place  their  experience  in  a more  diverse  context  and  disregarded  the  problems  different  from  their 
own.  Also,  due  to  the  lack  of  classroom  experience  as  teachers,  they  tended  to  recall  critical  prior  experiences  as  a 
student  to  plan  their  lessons,  to  predict  students’  response,  and  to  evaluate  certain  teaching  strategies  (Holt-Reynolds, 
1992). 

In  addition  to  relying  on  personal  history,  this  study  showed  that  the  preservice  teachers  also  used  inference 
strategies  to  design  technology-integrated  learning  activities  that  were  not  available  from  their  own  experience.  They 
used  activity-based  inference  to  modify  and  extend  previous  learning  activities  by  incorporating  technology.  They 
also  used  tool -based  inference  to  create  original  learning  activities  by  matching  technology's  affordances  with  the 
need  of  their  teaching  situations.  With  these  inference  strategies,  the  preservice  teachers  were  likely  to  go  beyond 
directly  applying  self  learning  experience  to  lesson  design,  because  they  had  to  integrate  the  technology  elements 
that  may  give  rise  to  fundamental  changes  in  the  lesson  structure.  However,  the  preservice  teachers'  utilization  of 
both  prior  experience  and  inference  strategies  were  still  supported  by  their  established  beliefs  and  assumptions 
generalized  from  personal  history. 

Scaffolding  Divergent  and  Critical  Thinking  in  Technology  Integration 

Educational  technology  courses  in  teacher  preparation  programs  have  been  criticized  for  teaching  standalone 
technology  skills  rather  than  cultivating  preservice  teachers’  ability  of  effective  technology  integration  (Gunter, 

2001;  Whetstone  & Carr-Chellman,  2001).  Aside  from  incorporating  technology  in  method  courses  and  field 
teaching,  some  teacher  preparation  programs  have  started  project-based  learning  in  educational  technology  courses. 
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in  which  the  preservice  teachers  have  chance  to  design  technology-infused  lessons  (e.g.  McRobbie  et  al.,  2000).  Ill- 
structured  design  problem  solving  activities  were  involved  as  well,  but  they  were  not  situated  in  a problem-based 
environment  and  did  not  emphasize  the  systematic  problem-solving  process  and  the  related  thinking  skills. 
Meanwhile,  according  to  literature,  problem  solving  entails  a critical  creative  process  that  involves  divergent 
thinking,  which  supports  creative  ideation,  and  critical  thinking,  which  ensures  goal-directed  actions  (Mayer  & 
Wittrock,  2006;  Runco  2003).  This  viewpoint  fits  well  with  the  design  aspect  of  technology  integration  problems. 
However,  divergent  and  critical  thinking  is  rarely  mentioned  in  the  project-based  learning  of  technology  integration. 

This  study  redesigned  project-based  learning  by  creating  a problem  scenario  and  scaffolding  the  problem 
solving  process  with  a focus  on  divergent  and  critical  thinking,  hoping  to  provide  a situated  context  and  more 
guidance  for  preservice  teachers  to  integrate  technology.  Findings  from  this  study  indicated  that  this  was  a beneficial 
way  for  preparing  preservice  teachers  to  teach  with  technology.  The  problem-based  teaching  scenario  together  with 
the  divergent  thinking  scaffolds  promoted  the  preservice  teachers  to  consider  different  constraints  and  challenges 
before  integrating  technology  and  to  brainstorm  various  solutions  by  inquiring  into  the  possible  combinations  of 
technology,  pedagogy,  and  content.  The  critical  thinking  scaffolds  facilitated  them  to  focus  on  the  important 
problems  to  be  solved,  develop  criteria  for  evaluating  the  divergent  solutions,  as  well  as  to  monitor  and  adjust  their 
lesson  design.  Divergent  and  critical  thinking  scaffolds  not  only  enhanced  the  preservice  teachers'  performance  but 
also  exposed  them  to  the  complexity  behind  solving  technology  integration  problems,  from  which  they  began  to 
develop  new  understandings  and  skills  of  teaching  with  technology. 

Nevertheless,  problems  arising  from  this  study  should  also  be  pointed  out.  Because  of  the  preservice 
teachers'  characteristics  as  novices,  their  assumptions  and  strategies  led  them  toward  convergent  and  uncritical 
thinking,  inhibiting  the  intended  effects  of  the  problem  situation  and  the  scaffolds.  Although  they  reported  new 
perceptions  about  lesson  design  after  completing  the  projects,  their  assumptions  and  strategies  remained  unchanged 
and  were  not  integrated  with  the  new  understandings  emerged  from  using  the  scaffolds.  In  addition,  some  of  the 
preservice  teachers  failed  to  recognize  the  purpose  of  divergent  and  critical  thinking,  resulting  in  the  lack  of 
motivation  and  misuse  of  some  scaffolding  steps.  Future  research  questions  may  include  how  to  improve  the 
scaffolds  in  ways  that  can  encourage  the  preservice  teachers  to  critically  reflect  on  their  underlying  assumptions, 
how  to  make  them  understand  the  point  of  using  divergent  and  critical  thinking  in  solving  technology  integration 
problems,  and  how  to  help  them  develop  new  strategies  that  can  facilitate  divergent  and  critical  thinking. 

This  study  contributed  to  the  field  of  Preparing  Tomorrow’s  Teachers  to  use  Technology  (PT3)  by 
suggesting  a new  strategy  of  teaching  educational  technology  course,  which  paves  the  way  for  the  preservice 
teachers  to  use  technology  in  their  method  courses,  field  teaching,  and  future  career.  Teacher  educators  in  the  field 
of  educational  technology  may  experiment  with  the  guided  problem  solving  approach  when  using  project-based 
learning  in  their  class,  so  as  to  provide  more  structure  for  the  novice  lesson  designers  and  nurture  the  desirable 
thinking  skills.  Teacher  educators  who  wish  to  integrate  technology  into  method  courses  or  field  teaching  may  also 
use  the  divergent  and  critical  thinking  scaffolds  to  help  preservice  teachers  plan  for  incorporating  technology  into 
the  pedagogies  and  content  they  have  learned.  This  study  may  also  inform  educational  researchers  interested  in  PT3 
about  the  possibilities  and  challenges  in  developing  preservice  teachers’  technological  pedagogical  content 
knowledge. 


Conclusion 

The  major  purpose  of  this  study  was  to  examine  the  preservice  teachers’  experience  and  perceptions  about 
the  guided  problem  solving  process  in  order  to  understand  the  appropriateness  of  this  approach  in  preparing  them  to 
teach  with  technology.  Developing  lessons  effectively  incorporating  technology  tools  is  an  ill-structured  design 
problem  challenging  for  preservice  teachers.  Problem-solving  scaffolds  with  an  emphasis  on  divergent  and  critical 
thinking  have  the  potential  for  developing  preservice  teachers’  knowledge  and  skills  in  designing  technology- 
integrated  lessons.  Problems  and  limitations  found  in  this  study  reminded  the  researcher  of  the  preservice  teachers’ 
characteristics  in  learning  and  also  indicated  directions  for  improving  the  scaffolds.  In  conclusion,  the  guided 
problem  solving  approach  is  worthwhile  for  further  study  and  will  become  a useful  strategy  contributing  to  the 
research  and  practice  in  the  PT3  field. 
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Appendix  A:  Sample  Case  and  Scaffolds 


Communication  and  Collaboration 

The  grant  your  school  applied  for  was  awarded  and  you  have  access  to  several  new  computers  in  your  classroom. 
You  are  excited  about  the  computers  and  want  to  use  them  to  support  students'  communication  and  collaboration, 
which  are  important  skills  for  your  students  to  have.  From  your  previous  experience,  you  know  that  students  are 
more  motivated  to  learn  when  they  are  given  the  opportunity  to  collaborate  in  groups  and  communicate  their  ideas  to 
a "real  audience".  More  importantly,  they  can  help  each  other  with  difficulties  and  negotiate  conflicting  opinions 
though  collaboration,  which  leads  to  deeper  understanding  of  course  content. 

However,  you  remember  that  things  don't  always  go  smoothly  when  your  class  does  collaborative  projects.  You 
remember  that  during  the  last  collaborative  learning  project,  students  seemed  to  be  actively  talking  to  each  other,  but 
not  everyone  learned  as  much  as  they  were  expected  to  due  to  a number  of  problems.  For  example,  some  students 
contributed  a lot  and  even  dominated  the  teamwork,  while  some  students  were  not  truly  engaged  and  kept  talking 
about  irrelevant  things.  Some  groups  of  students  really  seemed  to  respond  well  from  the  activity,  while  others 
seemed  to  learn  very  little.  While  negotiating  different  ideas,  some  students  failed  to  listen  to  each  other  carefully 
before  they  gave  responses,  which  resulted  in  unpleasant  collaboration  that  ended  up  with  conflicts,  disagreements, 
and  even  quarrels. 

The  current  curriculum  standard  you  are  planning  to  teach  can  best  be  achieved  by  using  teaching  strategies  that 
emphasize  peer  interaction  and  collaborative  learning.  You  know  from  your  EDIT  2000  class  that  technology  can 
provide  your  students  with  rich  learning  resources,  productivity  tools,  and  communication  tools.  You  decide  to 
integrate  technology  with  teaching  strategies  to  address  the  problems  you  have  experienced  with  communication  and 
collaboration. 

Step  1:  Select  Curriculum  Standard 

Select  a curriculum  standard  from  your  subject/grade  level  that  involves  or  requires  communication  and 
collaboration. 

1)  What  curriculum  standard  will  you  address  in  this  lesson? 

2)  Why  did  you  choose  this  curriculum  standard? 


Step  2:  Identify  the  Challenges 

Situate  your  the  standard  that  you  are  going  to  teach  into  the  case  scenario  and  think  about  what  specific  challenges, 
in  terms  of  communication  and  collaboration,  you  might  meet  in  teaching  this  standard.  List  your  challenges  in  the 
following  area  (Number  each  challenge). 


Step  3:  Frame  the  Instructional  Problem 

Based  on  the  challenges  you  listed  in  Step  2,  which  one  is  the  biggest  issue?  This  will  be  your  instructional  problem. 
Describe  the  problem  that  you  want  to  solve  in  this  case  project.  Make  sure  your  instructional  problem  is  relevant  to 
communication  and  collaboration  and  could  benefit  from  technology  integrated  teaching  and  learning. 

1)  What  is  your  instructional  problem? 

2)  Why  is  this  instructional  problem  important? 


Step  4:  Generate  Solutions 

Think  about  how  you  might  incorporate  technology  into  teaching  and  learning  to  solve  your  instructional  problem. 
Review  Chapter  3 in  your  textbook  to  learn  about  communication  and  collaboration  and  the  sample  lessons  while 
you  are  working  on  this  step.  Consider  the  different  technological  tools  you  have  learned  either  in  EDIT  2000  or  by 
yourself  and  brainstorm  as  many  solution  ideas  as  you  can.  List  your  solutions  in  the  following  area  (Number  each 
solution). 
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Step  5:  Select  Criteria 

Your  task  now  is  to  select  the  most  promising  solutions  from  your  Step  4.  Generate  criteria  that  will  help  you 
determine  the  appropriateness  of  your  solutions.  Select  and  list  FIVE  criteria  that  you  think  are  most  relevant  and 
important  for  evaluating  your  solutions.  Each  criterion  should  have  a different  focus. 

Five  Criteria 


1 

2 

3 

4 

5 

Step  6:  Apply  Criteria 

Apply  the  criteria  to  your  solutions  for  the  Instructional  Problem.  Use  each  criterion  to  rank  the  solutions  in  an 
evaluation  matrix.  Use  a scale  from  1 (poorest)  to  10  (best)  and  enter  the  numbers  in  the  appropriate  columns.  Add 
the  ranks  you  have  given  to  each  solution  and  enter  the  sums  of  the  TOTAL  column. 


Solutions 

(Summarize  your  solutions  ) 

Criteria 

1 2 3 4 5 

Total 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 
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Step  7:  Design  your  Lesson 

Turn  your  most  promising  solution  into  a practical  lesson  plan.  Explain  more  specifically  how  you  will  implement 
your  lesson,  how  you  will  use  technology  to  support  communication  and  collaboration,  and  why  your  lesson  activity 
will  solve  your  the  instructional  problem.  Here  is  the  information  you  should  include: 

Lesson  Title 
Grade/Content  Area 
Lesson  Duration 

*State  Objectives  (these  will  come  directly  from  the  Georgia  Performance  Standards) 

*Select  Tools,  Materials,  and  Teaching  Methods  (what  will  you  use  to  accomplish  this  learning  activity?) 

*List  Student  Task/Product  (what  is  the  specific  task  you  want  students  to  complete?  This  should  be  no  more  than  3 
sentences). 

*Describe  Learner  Participation  (describe  the  steps  of  the  learning  activity.  How  will  students  be  divided  into  groups 
with  specific  responsibilities,  how  will  the  lesson  be  introduced,  how  will  you  monitor  student  learning  during  the 
activity,  what  will  students  be  doing  during  the  activity?) 

*Evaluate  Learner  Outcomes  (describe  how  you  will  know  students  have  achieved  your  stated  learning  objectives). 
*Create  a Student  Sample  (Provide  one  sample  that  you  expect  your  students  would  create  in  this  lesson.) 


Step  8:  Reflection 

Take  a moment  to  reflect  on  your  performance  in  case  one.  This  helps  you  learn  from  your  experience  and  do  a 
better  job  in  case  two. 

1)  As  I worked  on  case  one,  I did  well  in... 

2)  As  I worked  on  case  one,  I had  difficulties  with... 

3)  As  I worked  on  case  one,  I wish  I had  spent  more  time  on... 

4)  To  do  a better  job  in  case  two,  I need  to... 
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Abstract:  The  selection  of  educational  technology  to  support  student  learning  in  higher  education  is  often 
implemented  as  stand  alone  solutions  to  individual  problems  within  a university.  A systems  view  of  the 
university’s  enterprise  systems  can  reduce  silos  of  technology,  and  that  provide  economy  of  scale,  both  in  support 
costs,  as  well  as  in  training  and  management.  Acting  as  an  action  researcher,  with  the  role  as  an  internal 
consultant,  a technology  integrator  can  help  identify  efficiencies  and  solutions  to  common  problems  faced  by 
instructors  and  institutions.  Building  support  structures  for  faculty  to  support  the  adoption  of  technology,  as  well 
as  encouraging  individual  adoption  and  evaluation  of  technology  by  faculty,  can  help  faculty  determine  the 
usefulness  of  technology  in  education. 

Key  words:  Transformational  Change,  Management 

Introduction 

Higher  education  institutions  have  moved  away  from  simple  on-boarding  of  a variety  of  individual  tools 
and  systems,  and  an  enterprise  approach  has  begun  to  take  shape  at  many  institutions.  Guided  by  environmental 
pressures,  institutions  are  seeking  to  find  efficiencies  in  technology  support  and  deployment  on  campuses. 
Combined  with  internal  or  governance  pressure  to  link  expenditures  to  strategic  initiatives  helps  to  encourage  a 
shift  to  a high  level  perspective  of  technology  integration  at  institutions.  As  organizations  are  shaped  and  adapt  to 
these  new  pressures,  terminology  and  methodology  for  describing  and  measuring  this  change  need  to  be  used  to 
inform  the  discussion.  Existing  models  for  discussing  these  changes  can  be  found  in  the  literature  of 
organizational  development  and  information  systems.  As  technology  integrators  seek  to  move  their  practice 
towards  a more  strategic  focus  within  their  institutions,  it  is  worthwhile  to  consider  the  concepts  from  these  fields. 

Stages  of  growth 

Nolan  (1979)  described  six  stages  of  data  processing  growth  within  businesses:  I)  Initiation,  II) 
Contagion,  III)  Control,  IV)  Integration,  V)  Data  Administration  VI)  Maturity.  While  Nolan’s  Stages  of  Data 
Processing  Growth  was  developed  based  on  experience  with  data  processing  within  businesses,  it  provides  a 
useful  framework  by  which  to  explore  an  organization’s  maturity  with  technology  utilization.  The  model  has  been 
particularly  influential  in  the  field  of  information  systems,  and  while  it  has  some  detractors,  it  does  provide  a 
method  by  which  to  consider  the  role  that  the  organizational  environment,  and  organizational  learning  play  in 
technology  utilization  in  a particular  context.  If  you  substitute  the  innovation  of  data  processing  in  an  organization 
with  the  use  of  educational  technology  in  higher  education,  Nolan's  model  provides  some  affordances  for 
describing  the  current  state  of  technology  integration  in  our  institutions.  If  you  consider  the  six  stages  and  Nolan's 
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description,  you  will  find  surprising  similarities  to  the  challenges  facing  higher  education  institutions  as 
technology  systems  evolve  within  their  organizations.  Stage  I:  Initiation  is  comprised  of  the  introduction  of 
several  systems  in  a particular  group  within  the  organization.  Nolan  identifies  accounting  as  the  point  of  entry  for 
most  of  these  systems.  In  stage  one,  users  are  not  very  aware  of  the  technology  throughout  the  organization.  Stage 
2:  Contagion  consists  of  growth  and  expansion  of  systems  to  other  areas  of  the  organization,  with  little  centralized 
control  in  place  to  keep  costs  under  control.  Stage  III:  Control  is  reached  in  part  due  to  frustration  with  the 
explosion  of  costs  and  systems  in  stage  II.  In  stage  III  cost  control  measures,  professionalization  of  the  systems 
support  activities  and  end  user  accountability  for  systems  costs.  Stage  IV:  Integration  is  reached  when  the  systems 
and  applications  provided  to  users  are  reliably  provided  and  well  organized.  This  capacity  allows  end  users  to  take 
greater  advantage  of  the  systems  in  their  activities,  and  real  value  is  gained  from  use  of  the  systems.  Growth  in  the 
number  of  systems  and  applications  occurs.  Stage  V : Data  Administration  consists  of  the  containment  of  demand 
from  stage  IV  and  Stage  VI:  Maturity  represents  a balanced  supply  and  demand  for  systems.  (Nolan,  1979)  When 
viewed  through  the  stages  of  growth  model,  the  adoption  of  the  use  of  educational  technology  within  an  institution 
can  be  categorized  by  these  stages.  Stage  I would  consist  of  a single  department  or  a group  of  faculty  adopting  a 
particular  technology,  followed  by  the  proliferation  of  the  technology  to  multiple  departments  on  a campus  in 
stage  2.  As  a cost  saving  measure,  and  to  reduce  inefficiencies,  the  technology  support  may  be  centralized  or 
placed  under  strong  controls  by  a professional  technology  group  within  the  institution  in  stage  3.  Stage  4 would 
consist  of  a view  of  technology  services  as  application  portfolio  based,  the  development  of  service  level 
agreements  within  the  institution,  and  planning  and  control  to  meet  each  user  group's  needs.  Stage  4 would  lead 
towards  stage  5 when  shared  data  and  systems,  along  with  application  integration  is  realized,  leading  to  the  mature 
support  model  in  stage  VI.  From  a pragmatic  perspective,  you  could  consider  stage  1 technologies  to  be  comprised 
of  tools  that  individual  faculty  utilize  within  their  institutions,  such  as  media  presentation  technology,  in-class 
activities,  and  systems  used  to  support  a limited  customer  base  within  the  organization.  Stage  2 technologies 
would  include  things  such  as  shared  technology  resources,  such  as  a computer  lab,  a shared  classroom  equipped 
with  technology,  or  a shared  server.  Systems  such  as  course  management  systems  appear  to  fit  the  model  of  the 
kinds  of  systems  that  would  appear  somewhere  around  stage  3 or  4 according  to  the  stages  of  growth  model.  These 
systems  may  become  integrated  with  the  institution’s  student  information  systems,  a campus  portal,  or  other 
unifying  technologies  around  stage  4.  According  to  McConachie  and  Danaher  (2005),  many  institutions  have 
moved  towards  the  integration  stage  for  the  deployment  and  support  of  technology  systems  on  campuses,  in  order 
to  reduce  costs,  and  to  provide  efficiencies.  Stage  4 appears  to  be  a common  stage  for  many  higher  education 
institutions  at  the  time  of  this  paper.  While  the  stages  of  growth  model  is  a simplification  of  a complex 
phenomenon  of  technology  adoption  in  an  organization,  it  at  least  provides  a method  by  which  to  conceptualize 
the  learning  on  both  the  part  of  the  technology  support  functions  of  an  organization,  as  well  as  the  organization  as 
a whole. 


Role  of  a technology  integrator 

When  moving  from  the  support  of  single  systems  or  technologies,  and  towards  a systems  perspective 
there  is  a change  in  the  role  of  technology  professionals  who  assist  others  with  technology  integration. 
Historically,  a large  portion  of  what  educational  technology  professionals  have  done  in  higher  education  has  been 
training  and  the  development  of  support  resources  for  the  use  of  educational  technology  tools.  While  individual 
technologies  may  provide  solutions  for  individual  problems,  or  meet  the  needs  of  a particular  context,  the 
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perspective  value  of  the  technology  investment  and  subsequent  training  must  be  considered  valuable  by  the  host 
institution  or  continued  support  of  the  activity  may  diminish.  The  move  from  training  to  performance  leads  to  a 
more  strategic  approach  for  professional  activity,  and  includes  the  need  to  work  collaboratively  and  consider  the 
readiness  of  the  organization.  (Rossett,  2002).  When  considering  the  strategic  context  of  a particular  technology  or 
a portfolio  of  services,  the  analysis  needs  to  be  framed  from  an  organizational  perspective.  This  requires  that  the 
scope  of  the  analysis  must  be  adjusted  to  include  broad  organizational  variables,  as  well  as  include  an 
understanding  of  the  strategic  directions  of  the  institution.  This  contextual  review  allows  for  the  development  of 
resources,  including  training,  that  allow  faculty  to  focus  on  how  the  technology  will  allow  them  to  solve  problems 
either  in  their  face  to  face  courses,  or  those  caused  by  being  separated  from  their  students  by  either  space  or  time. 
When  the  level  of  analysis  is  broadened,  there  may  be  additional  challenges  or  variables  within  the  organization 
that  may  be  identified  as  barriers  for  reaching  the  desired  performance.  As  professionals  in  a performance  focused 
world,  a systems  view  of  higher  education  institutions  can  assist  with  analysis  and  performance  diagnosis. 
Furthermore,  the  tools  and  literature  of  organizational  development  provide  a framework  for  investigating  the 
variables  that  may  inhibit  change.  The  utilization  of  these  tools  require  both  access  and  capability  to  enact  change 
or  advocate  for  change  where  needed.  Facilitating  the  development  of  efficient  and  effective  solutions  for 
technology  integration  that  are  designed  to  support  faculty  across  the  entire  institution  supports  the  evolution  of 
institutions  from  Nolan  level  3 institutions  to  Nolan  level  4 institutions.  The  development  of  reliable  and  cost 
effective  solutions  to  meet  the  needs  of  multiple  groups  of  faculty  within  an  institution  can  help  further  the 
adoption  and  growth  of  the  solutions  within  an  institutions.  If  these  solutions  fit  closely  with  faculty 
responsibilities  and  needs,  as  well  as  institutional  strategies,  then  both  the  systems  and  the  social  systems  are 
closely  aligned.  The  use  of  integrated  systems  that  interface  with  existing  student  information  systems,  and  are 
common  across  students'  experience,  allow  for  the  technology  to  become  a facilitator,  rather  than  a distraction  for 
instruction.  New  educational  systems  provide  greater  opportunities  to  rapidly  categorize,  capture,  and  access 
instructional  content  with  a greater  degree  of  automation.  These  tools  can  help  reduce  the  barriers  to  entry  and 
improve  the  relative  advantage  perceived  by  the  faculty  considering  adopting  a new  technology.  As  institutions 
identify  new  technologies  for  faculty  to  use  in  the  support  of  their  instruction,  a balance  between  individual  faculty 
freedom  of  control  over  the  conditions  of  the  classroom  technology  and  the  need  for  the  technology  administrators 
to  provide  reliable  support  for  the  broader  campus.  These  competing  interests  provide  for  conflict  within 
institutions  of  higher  education.  Faculty  often  view  technology  integration  within  their  teaching  as  a issue  of 
academic  freedom,  and  systems  control  imposed  by  technology  administrators  creates  mistrust  from  faculty,  and 
can  limit  individual  faculty  experimentation  The  balancing  and  managing  of  expectations  and  responsibilities  are 
continually  renegotiated  within  any  organization  using  new  forms  of  technology  in  the  operation  of  an  enterprise. 
Rogers  (1995)  Diffusion  of  Innovation  theory  provides  a model  for  the  individual  adoption  of  innovations.  This 
model  includes  a component  of  experimentation  and  evaluation  of  the  technology  by  faculty.  Providing  a safe 
harbor  for  faculty  to  experiment  with  technology  is  critical  for  their  individual  adoption  of  technology.  However, 
faculty  do  not  operate  independently  when  they  work  within  an  institutional  context.  The  environment  that  they 
are  given  to  integrate  technology  within  their  instructional  methods  contains  a mixture  of  policies,  rules,  and 
technical  limitations  that  restricts  the  freedom  of  faculty  to  freely  implement  systems  based  on  their  own  vision  of 
efficiency. 
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Online  learning  from  a systems  perspective 


When  institutions  do  become  focused  on  a strategic  change  in  direction  for  the  use  of  technology  within 
instruction,  such  as  the  case  of  online  learning,  the  process  by  which  the  rules  and  conditions  of  the  organizational 
operation  are  renegotiated  is  important.  One  method  by  which  organizations  can  review  the  nature  of  the 
education  environment  is  via  a model  called  parallel  learning  structure  (Bushe  and  Shani,  1991 ).  In  this  model  an 
internal  organizational  structure  is  created  that  helps  the  organization  review  the  processes  and  conditions  of 
production  from  a new  perspective.  This  group  is  similar  to  the  traditional  model  of  committees  that  are  used  to 
discuss  changes  to  policies  or  particular  issues  on  a university  campus,  but  for  distance  education,  this  includes 
additional  functional  representatives  from  administrative  and  service  organizations  in  the  university.  These  other 
groups  involved  in  the  process  include  the  student  services,  technology  support,  and  general  administrative 
support  areas  that  form  the  processes  of  the  organization’s  business  function.  As  the  parallel  learning  structure 
discusses  and  plans  out  the  organization’s  operation  of  distance  learning  on  the  campus,  the  group?s  ability  to 
identify  and  address  process  issues  grows.  The  creation  of  a separate  functional  group  from  a cross  section  of  the 
campus  provides  an  opportunity  for  these  individuals  with  varying  areas  of  expertise  to  consider  the  immediate 
issue  of  distance  learning,  outside  of  the  normal  distractions  of  daily  business  operation.  The  result  of  such  a 
process  is  the  ability  of  the  institution  to  consider  and  develop  new  processes  of  operation  that  are  organized  based 
on  the  strategic  objective  of  distance  education. 


Conclusion 

Institutions  should  seek  to  develop  structures  and  roles  that  enable  organizational  learning. 
Organizational  learning  is  defined  by  Robey  et  al.  2000,  as:  "an  organizational  process,  both  intentional  and 
unintentional,  enabling  the  acquisition  of,  access  to,  and  revision  of  organizational  memory,  thereby  providing 
direction  to  organizational  action."  (p  130).  The  structures  that  support  this  capacity  to  learn  within  an  institution 
include  structures  such  as  parallel  learning  structures,  faculty  teaching  and  learning  centers,  as  well  as  internal 
consultants  that  help  facilitate  faculty  adoption  of  new  technology.  Utilizing  a mixture  of  systems  theory, 
information  systems  perspectives,  organizational  development  methodology,  and  a focus  on  performance  allows 
for  the  analysis  of  problems  outside  the  limited  scope  of  training  development.  This  focus  on  the  entire  system, 
along  with  a strategic  focus  for  activities  allows  for  the  alignment  of  development  activities  with  the 
organization’s  goals,  and  most  likely,  funding.  While  researching  how  changes  occur  within  a higher  education 
institution  at  a high  level  is  complicated,  research  in  this  area  is  necessary  if  we  are  to  realize  strategic  goals  for 
technology  integration  within  our  institutions  of  higher  education,  rather  than  isolated  competencies  in  particular 
technology  systems  or  individual  technologies. 


446 


References 


Bushe,  G.  R.  & Shani,  A.B.  (1991).  Parallel  learning  structures.  Reading  Massachusetts:  Addison  Wesley. 
McConachie,  J.,  & Danaher,  A.  (2005).  Central  Queensland  University's  course  management  systems: 

Accelerator  or  brake  in  engaging  change?  International  Review  of  Research  in  Open  and  Distance  Learning, 
6(1),  1-17. 

Nolan,  R.  (1979).  Managing  the  crises  in  data  processing.  Harvard  Business  Review,  57(2),  1 15-116. 

Robey,  D.,  Boudreau,  M.,  & Rose,  G.  (2000)  Information  technology  and  organizational  learning:  a review  and 
assessment  of  research.  Accounting,  management,  and  information  technologies  10(2).  125-155. 

Rogers,  E.  M.  (1995).  Diffusion  of  innovations.  (4th  ed.)  New  York:  the  Free  Press. 

Rossett,  A.  (2002).  From  training  to  training  and  performance.  In  R.  Reiser  & J.  Dempsey  (Eds.),  Trends  and 
Issues  in  Instructional  Design  and  Technology,  (pp.  123-132).  Columbus,  Ohio:  Merrill/Prentice  Hall. 


447 


What  Is  The  Real  Situation  In  Practice  Despite  The  Many  Action  Plans 
Implemented  For  Technology  Integration  Process  In  Education? 

Ercan  Top 

ercantop  @ gmail.com 
Abant  Izzet  Baysal  University,  Turkey 

Zahide  Yildirim 
zahidey  @ metu . edu.tr 
Middle  East  Technical  University,  Turkey 

Aysegul  Daloglu 
daloglu@metu.edu.tr 
Middle  East  Technical  University,  Turkey 

Abstract:  There  is  a trend  in  Turkish  educational  system  to  integrate  technology  into  all  levels  of 
education.  For  example,  with  the  admission  of  the  Basic  Education  Law  in  1997,  computer  and 
instructional  technology  related  courses  have  been  included  in  teacher  education  curricula  since 
1998  in  Turkey.  The  main  aim  of  this  study  was  to  investigate  the  secondary  school  English 
teachers’  perceptions  of  technology  and  technology  integration  processes  into  their  lessons. 
Graduating  in  2002  or  after  from  teacher  education  institution  was  a criterion  for  selection  of  these 
teachers.  The  purpose  of  using  this  criterion  was  to  investigate  the  technology  perceptions  and 
technology  integration  of  teachers  who  had  taken  computer  and  instructional  technology  related 
courses  in  their  undergraduate  education. 

1.  Introduction 

Parallel  to  Worldwide  developments,  through  the  ‘National  Education  Development  Project’,  the  Higher 
Education  Council  (HEC)  in  Turkey  redesigned  the  teacher  education  programs’  curricula  to  improve  the  quality  of 
and  to  integrate  ICT  into  teacher  education  programs  (Akkoyunlu  & Orhan,  2001).  One  of  the  implications  of  this 
project  was  to  integrate  computer  and  instructional  technology  related  courses  into  teacher  education  curricula 
(HEC,  1998).  By  integrating  these  courses,  HEC  were  planning  to  have  teachers  who  "know  the  technology  and 
apply  it  efficiently  in  instructional  settings"  (Akkoyunlu  & Orhan,  2001,  p.  30-31). 

Many  actions  have  been  taken  since  1984  to  the  present  to  integrate  technology  in  instructional 
environments  in  Turkey.  Some  of  the  efforts  are  as  follows; 

• To  keep  up  with  the  latest  trends  in  education  MoNE  started  a project  by  establishing  a committee  related 
to  computer  education  in  1984.  . In  accordance  with  the  committee's  report,  MoNE  supplied  100  schools 
from  67  cities  with  hardware,  and  provided  computer  literacy  and  basic  programming  language  training 
programs  for  the  teachers  during  the  same  year. 

• In  1987,  context  of  the  training  was  expanded,  and  computer-aided  instruction  was  included  for  the  first 
time  in  teacher  training  programs. 

• In  1990,  MoNE  decided  to  include  computer  related  courses  in  the  curriculum  of  teacher  training 
institutions. 

• Since  1991,  MoNE  has  been  working  in  cooperation  with  universities  and  TUBITAK  (The  Scientific  and 
Technological  Research  Council  of  Turkey)  to  integrate  educational  technology  into  instruction. 

• In  1998,  with  the  acceptance  of  the  Basic  Education  Law,  computer  related  courses  in  teacher  training 
institutions  were  implemented  and  the  compulsory  education  in  Turkey  was  extended  to  eight  years. 

• In  1998,  under  the  supervision  of  MoNE,  General  Directorate  of  Educational  Technologies  (EgiTek)  was 
founded  to  carry  out  the  necessary  issues  in  developing  and  producing  any  kind  of  audio  visual  and 
computer  based  materials  to  use  in  educational  settings  (Egitek,  n.d.). 

• In  2003,  an  arrangement  was  made  for  20.000  schools'  Internet  connection  with  an  Internet  service 
provider.  At  the  end  of  2008,  there  were  high  band  ADSL  connections  in  33.018  schools/institutions  and 
wireless  internet  connections  in  4.870  schools/institutions  (MEB,  2009). 
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• In  2006,  HEC  formed  a study  group  (constituting  25  academicians)  to  refine  the  teacher  education  curricula 
designed  in  1998  According  to  this  group  reports,  HEC  (2006)  promulgated  changes  in  teacher  education 
programs. 

• In  2009,  100  % of  the  high  schools  and  94  % of  the  primary  schools  with  approximately  12  million 
students  had  access  to  Internet  with  621.000  computers  in  Turkey  (MEB,  2009). 

There  are  many  research  studies  related  with  the  teachers  usage  of  technology  in  education  in  Turkey.  For 
example.  Top's  (2003)  study  showed  that  preservice  teachers’  perceptions  of  their  competencies  with  regard  to 
technology  standard  were  found  quite  high.  Similarly,  Toker's  (2004)  study  indicated  that,  most  of  the  pre-service 
teachers  see  themselves  as  intermediate  technology  users.  On  the  other  hand,  Akkoyunlu  and  Kurbanoglu  (2004) 
carried  out  a study  indicating  that  teachers’  information  literacy  self-efficacy  level  was  generally  low.  In  another 
study,  Usluel  and  Haslaman  (2003)  revealed  that  teachers'  present  situation  scores  were  lower  than  their  preferred 
situation  scores  related  to  computer  technology  usage,  impact  on  students  and  purpose  of  usage.  Correspondingly, 
Askar  and  Usluel  (2003)  showed  that  teachers  favored  to  use  computers  for  administrative  purposes  rather  than 
educational  purposes.  Moreover,  Askar  and  Usluel  (2002)  stated  that,  teachers  found  computers  as  advantageous, 
compatible,  and  observable  in  management  and  personal  issues.  On  the  other  hand,  teachers  have  doubt  in  using 
computers  as  an  instructional  tool.  Similarly,  Cagiltay  et  al.  (2001)  conducted  a study  on  teachers’  perspectives 
about  the  use  of  computers  in  education.  They  find  out  that  teachers  have  positive  beliefs  on  the  usage  of  computers 
in  classes  and  many  of  the  teachers  desire  to  learn  things  related  to  the  usage  of  computers  in  classes. 

Although  there  are  many  research  studies  related  with  teachers’  technology  skills  and  knowledge  in 
Turkey,  the  field  lacks  of  the  research  study  which  is  conducted  especially  to  investigate  in-service  English 
teachers’,  who  graduated  from  the  faculty  of  education  and  took  the  compulsory  computer  and  instructional 
technology  related  courses  in  their  teacher  education  period,  instructional  technology  usage  in  their  teaching  and 
professional  environment.  Additionally,  there  is  no  study  that  that  focuses  on  the  observation  and  interview  as 
analysis  of  these  teachers  could  be  found  in  the  literature.  It  can  be  said  that  there  is  a need  to  reveal  these  teachers 
technology  usage  in  their  teaching  environment. 

The  purpose  of  this  study  is  ( 1)  to  reveal  how  high  school  English  teachers  perceive  variety  of  instructional 
technologies,  and  (2)  to  investigate  how  high  school  English  teachers  integrate  instructional  technology  into  their 
courses.  The  following  research  questions  guided  this  study: 

• What  are  high  school  English  teachers'  perceived  competency  levels  in  instructional  technology? 

• How  do  they  plan  to  use  or  integrate  instructional  technologies  in  their  courses? 

• For  what  purposes  and  how  do  they  use  instructional  technologies  in  their  courses? 

• How  and  for  what  purposes  do  they  use  instructional  technologies  in  assessment  and  evaluation  process? 

• How  do  they  use  technology  for  professional  development  purpose? 

• What  do  they  consider  about  social,  ethical,  legal,  and  human  issues  while  using  instructional  technologies? 

2.  Research  Design  and  Method 

Qualitative  approach  is  used  in  this  study.  Observations,  document  analysis,  and  interviews  were  used  to 
collect  the  data.  Graduating  2002  or  after  from  the  university  was  a criterion  for  the  selection  of  the  teachers 
participated  in  this  study.  The  aim  of  having  this  criterion  was  choosing  teachers  who  had  taken  the  computer  and 
instructional  technology  related  courses  during  their  undergraduate  education.  To  be  able  to  represent  the  population 
as  much  as  possible,  another  criterion  used  in  selection  of  the  participants  was  the  schools  types  (private,  Anatolian, 
vocational,  regular  high  schools)  that  the  teachers  were  teaching  in.  For  this  study,  from  four  types  high  schools 
(private,  Anatolian,  vocational,  and  regular  high  schools)  ),  17  high  schools,  and  an  English  teacher  from  each 
selected  school  were  chosen  in  Ankara,  Turkey. 

The  data  were  collected  through  observation,  document  analysis  and  interviews.  Observations  were 
conducted  based  on  a pre-prepared  observation  guide.  Teachers  were  observed  in  their  courses  during  a whole  day. 
The  purpose  of  observation  was  to  describe  the  high  school  English  teachers’,  use  of  instructional  technologies  in 
their  courses  to  provide  triangulation  for  the  interview  data  and  to  provide  basis  for  the  interview  schedule 
development.  From  each  teacher,  their  course  annual/lesson  plans  were  obtained  to  clarify  their  instructional 
technology  knowledge,  to  show  how  they  plan  to  use  instructional  technologies  in  their  courses,  to  explain  how  they 
use  instructional  technologies  in  their  courses,  and  to  understand  what  they  consider  while  using  instructional 
technologies.  The  interview  protocol  was  developed  based  on  National  Educational  Technology  Standards  for 
Teachers  (NETS-T,  2000)  indicators,  and  consisted  of  eight  main  categories:  (1)  teachers’  educational  background, 


449 


(2)  teachers’  basic  technology  knowledge,  (3)  teachers'  knowledge  about  the  use  of  technology  in  planning  and 
designing  learning  environments  and  experiences,  (4)  teachers'  knowledge  about  the  use  of  technology  in  teaching, 
learning,  and  the  curriculum,  (5)  teachers’  knowledge  about  the  application  of  technology  in  assessment  and 
evaluation,  (6)  teachers'  knowledge  about  the  ways  of  using  technology  for  their  productivity  and  professional 
practice,  (7)  teachers'  knowledge  about  the  use  of  technology  for  social,  ethical,  legal,  and  human  issues,  and  (8)  the 
accessibility  of  the  available  technologies  for  the  use  of  teachers  in  schools.  Interviews  were  conducted  on  a 
voluntary  basis  with  the  17  high  school  English  teachers.  The  interviews  were  semi-structured  in  nature.  Before  the 
interviews,  the  observation  notes  and  annual/lesson  plans  were  analyzed  to  be  able  to  ask  exploring  and  relevant 
questions. 

In  this  study,  basically  content  analysis  was  used  to  analyze  the  data.  The  interviews  were  transcribed, 
typed  and  coded  as  to  the  main  data  sources.  In  determining  the  coding  categories  the  NETS-T  (2000)  categories 
were  considered.  Before  creating  coding  schema,  all  of  the  studied  teachers  were  analyzed  by  using  this  standard’s 
indicators  by  finding  evidences  from  observation  data,  annual/lesson  plans,  and  interview  transcripts.  While 
categorizing  the  teachers.  International  Society  for  Technology  in  Education's  "General  Preparation  Performance 
Profile  Test”  (Kelly,  2002)  indicators  were  used. 

3.  Findings 

It  is  found  that,  even  though  the  investigated  teachers’  perceived  basic  technology  knowledge  level  could 
be  accepted  as  average,  they  have  different  levels  of  basic  technology  knowledge.  When  schools  are  compared, 
private  high  schools  teachers'  perceived  competency  level  is  higher  than  that  of  public  high  school  (Anatolian, 
regular,  and  vocational)  English  teachers.  At  the  same  time,  private  high  school  English  teachers  are  more  eager  to 
learn  things  on  current  and  emerging  technologies. 

The  study  showed  that  almost  all  of  the  teachers  included  in  the  study  have  enough  knowledge  on  where 
and  when  to  use  technology.  Almost  all  of  them  have  the  necessary  qualifications  to  plan  technology  usage  in  their 
classrooms  according  the  NETS-T’s  indicators. 

The  teachers'  use  of  technology  knowledge  were  investigated  by  using  NETS-T  “ Teaching , Learning,  and 
the  Curriculum ” indicators.  The  results  showed  that  while  private  high  school  English  teachers  did  not  have 
problems  on  most  of  these  indicators,  Anatolian  high  school  English  teachers  had  trivial  problems  and  regular  and 
vocational  high  school  English  teachers  had  severe  problems  on  some  of  these  indicators. 

The  findings  indicated  that  there  were  differences  in  technology  integration  between  private  and  public 
high  school  English  teachers  on  NETS-T  “ Evaluation  and  Assessment ” indicators.  Although,  three  out  of  four 
private  school  teachers  have  knowledge  on  these  indicators,  only  one  out  of  13  public  school  teacher  has  knowledge 
on  them. 

There  were  big  differences  on  the  NETS-T  “ Productivity  and  Professional  Practice ” indicators  between 
private  and  public  high  school  English  teachers  in  favor  of  private  high  school  teachers.  The  findings  of  the  study 
also  showed  that,  only  nine  out  of  17  teachers  stated  formal  education  contributed  to  their  professional  development 
in  integration  of  technologies. 

The  investigated  teachers  mentioned  various  purposes  of  using  technologies  in  English  teaching.  Some  of 
the  mentioned  purposes  are  as  follows  from  most  frequently  stated  to  least  frequently  stated;  to  provide  visual  help, 
to  get  students'  attention,  to  create  a learner  centered  environment,  to  enable  students  to  speak,  to  give  more  than  one 
stimulant,  to  enable  them  to  practice,  to  improve  students’  listening,  to  encourage  students,  to  develop  students’ 
pronunciation,  to  show  daily  usage  of  English,  and  to  have  audio  familiarity  with  some  language  items.  The 
mentioned  purposes  of  using  technologies  show  variations  among  school  types.  For  example,  “to  develop  students’ 
pronunciations”  and  “to  show  daily  usage  of  English”  were  stated  by  only  public  high  school  English  teachers  as 
purpose  of  using  technologies.  In  addition,  “to  enable  students  to  learn  by  doing”  were  stated  by  all  of  the  private 
high  school  English  teachers  and  only  two  of  the  13  public  high  school  English  teachers.  From  these  findings  it  can 
be  said  that  private  high  school  English  teachers  are  using  technologies  by  aiming  student-centered  teaching  and 
public  high  school  English  teachers  are  using  technologies  mainly  to  develop  students’  basic  facts  or  skills.  This  was 
partially  observed  during  the  observed  lessons,  and  also  the  public  high  school  English  teachers  accepted  that  they 
are  teaching  grammar  and  they  try  to  teach  basic  knowledge. 

In  this  study,  the  findings  also  showed  that  most  of  the  public  high  school  English  teachers  have 
inadequacy  of  knowledge  on  NETS-T’s  “Social,  Ethical,  Legal,  and  Human  Issues ” indicators.  However,  most  of 
the  private  high  school  English  teachers  have  lack  of  knowledge  on  only  one  these  five  indicators. 
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4.  Discussion 


Technical  skills  is  mainly  accepted  as  a necessary  first  step  in  moving  towards  using  technology  in 
educational  settings  in  national  technology  standards,  textbooks,  and  training  programs  for  teachers  (Sandholtz  & 
Reilly,  2004).  Even  though  the  investigated  teachers’  perceived  basic  technology  knowledge  level  could  be  accepted 
as  average,  they  have  different  levels  of  basic  technology  knowledge.  Indeed,  this  study  revealed  that  most  of  the 
public  high  school  English  teachers  included  in  this  study  were  in  need  of  developing  their  basic  technology 
operations  and  concepts.  Sandholtz  and  Reilly  (2004)  stated  that  “if  we  take  away  expectations  for  technical  skills 
and  allow  teachers  to  focus  on  developing  curriculum,  evaluating  learning  materials,  and  thinking  about  how  to 
provide  better  learning  opportunities  for  their  students,  teachers  are  likely  to  use  technology  more  effectively  and 
creatively  in  their  teaching”  (p.488).  Sandholtz  and  Reilly’s  proposal  may  work  under  the  conditions  where  teachers 
have  basic  computer  usage  competency.  In  this  study,  however,  there  were  teachers  who  even  did  not  use  e-mail  at 
all  or  rarely  used  it.  Given  that  these  teachers  have  taken  technology  courses  in  their  teacher  education  programs, 
they  could  be  expected  to  have  the  basic  technology  knowledge  but  that  is  not  the  case  for  some  of  them.  This  might 
indicate  that,  the  computer  and  instructional  technology  related  courses  might  not  be  as  effective  as  they  were 
planned. 

The  study  showed  that  almost  all  of  the  teachers  included  in  the  study  have  enough  knowledge  on  where 
and  when  to  use  technology.  Even  though  most  of  the  public  high  school  teachers  in  need  of  developing  their  basic 
technology  operations  and  concepts  knowledge,  almost  all  of  the  teachers  in  this  study  have  the  necessary 
qualifications  to  plan  technology  usage  in  their  classrooms  according  the  NETS-T’s  indicators.  From  these  findings 
it  might  be  said  that  the  studied  teachers  were  aware  of  the  use  of  educational  technologies  in  teaching  environment 
but  they  were  not  practicing  technologies  in  their  teaching.  One  of  the  reasons  for  that  situation  might  be  lack  of 
incentives,  enforcements  or  technological  infrastructure  to  use  technologies  in  their  teaching.  Technology  usage  in 
classrooms  requires  special  strategies  and  preparation  of  teachers  to  get  the  intended  benefit.  As  Becker  (2000) 
pointed  out,  the  exemplary  computer-user  teachers  are  preparing  themselves  to  use  computers  well  in  their  teaching. 
The  strategies  show  variations  depend  on  various  factors  such  as  the  used  educational  materials.  Similarly,  the 
teachers  included  in  the  study  also  mentioned  different  strategies  while  using  technologies  in  their  classrooms.  Some 
of  the  pointed  strategies  from  most  frequently  stated  to  least  frequently  stated  are  as  follows;  creating  student- 
centered  environment,  following  course  books,  using  intervals  for  elaborations,  creating  practice  opportunities  for 
students,  using  indirect  methods,  making  preparation  then  letting  students  perform,  considering  multiple  intelligence 
theory,  controlling  students’  understandings  through  applications,  and  using  the  available  sources. 

Tools  improve  our  cognition  and  the  current  technology  industry  provides  continuously  new  tools  (Young 
& Bush,  2004).  In  addition,  some  of  the  tools  have  proved  to  be  successful  and  effective  in  the  learning  contexts  and 
processes.  The  technologies  that  could  be  used  in  education  by  studied  teachers  are  general  tools  (cassette  player, 
CD  player,  projection,  video,  OHP,  etc...),  visuals,  and  audio-visual  materials.  However,  private  high  school 
English  teachers  mentioned  more  technologies  that  could  be  used  in  educational  settings  than  public  high  school 
English  teachers.  Only  six  of  the  investigated  teachers  mentioned  about  computer  applications.  One  reason  for  not 
mentioning  about  computer  applications  may  be  that  teachers  do  not  use  computers  in  their  lessons  and  personal  life 
much.  Another  reason  might  be  insufficient  technological  infrastructure  in  the  school. 

The  use  of  technologies  in  classroom  affects  students  on  various  ways.  For  example,  in  technology  rich 
classrooms,  students  become  more  motivated  (Schacter,  1999;  Ringstaff  & Kelley,  2002;  Kleiman,  20004),  become 
more  active  learners  (Ringstaff  & Kelley,  2002;  Kleiman,  2004),  have  better  confidence  (Ertmer  & Hruskocy,  1999; 
Ringstaff  & Kelley,  2002),  like  their  classes  more  (Schacter,  1999),  develop  more  positive  attitudes  (Schacter, 
1999),  have  better  self-esteem  (Ertmer  & Hruskocy,  1999),  improve  their  reflection  (Schacter,  1999),  learn  multiple 
perspectives  (Schacter,  1999),  increase  their  level  of  independent  thinking  (Schacter,  1999;  Ringstaff  & Kelley, 
2002),  improve  their  higher  order  thinking  skills  (Ringstaff  & Kelley,  2002),  improve  their  basic  computer  abilities 
(Ertmer  & Hruskocy,  1999;  Kleiman,  2004),  and  improve  their  achievement.  Similarly,  technology  usage  in  lessons 
has  various  effects  on  students  according  to  the  teachers  included  in  the  study.  The  effects  of  these  technologies  on 
students  are  generally  positive  and  in  accordance  with  the  related  literature.  Some  of  the  positive  effects  are; 
increasing  students’  motivation,  liking  the  usage  of  technology,  helping  students  to  develop  their  English,  giving 
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positive  responses,  becoming  more  active  in  lessons,  and  gaining  self  awareness.  Although  most  of  the  studied 
teachers  have  admitted  the  positive  effects  of  technology  usage  on  students  in  teaching  environment,  most  of  them 
especially  public  high  school  teachers  did  not  use  technologies  in  their  teaching. 

There  may  be  a great  number  of  excuses  stated  by  the  teachers  for  the  ignorance  of  the  use  of  technology  in 
teaching  environments.  For  example,  Ertmer  et  al.  (1999)  found  that  lack  of  equipment,  lack  of  time,  lack  of 
classroom  help,  lack  of  relevance,  mismatch  with  classroom  management  style,  and  lack  of  confidence  are  barriers 
to  technology  integration.  Similarly,  the  studied  teachers  (both  the  ones  who  use  technology  in  their  classroom  and 
the  ones  who  do  not)  also  put  forward  various  reasons  for  not  using  technologies  in  their  lessons.  For  instance  16  of 
17  teachers  emphasized  that  inadequacy  of  tools  in  classroom  and  time  were  the  reasons  for  not  using  technologies 
in  their  lessons.  Likewise,  Ringstaff  and  Kelley  (2002)  expressed  that  “without  sufficient  access  to  technology,  of 
course,  even  well-trained,  highly  motivated  teachers  will  not  be  able  to  integrate  technology  effectively  into 
instruction”  (p.  1 7).  Becker  (2000)  also  found  insufficient  access  and  not  having  enough  instructional  software  as 
major  reasons  for  not  using  technologies  in  educational  settings. 

Some  of  the  other  stated  reasons  from  the  most  frequently  stated  to  the  less  frequently  stated  are; 
inadequacy  of  contemporary  tools,  overloaded  curriculum,  difficulty  in  classroom  management,  feeling  difficulty  of 
arranging  available  technologies,  unsuitability  of  the  students  level,  difficulty  in  finding  suitable  materials,  students’ 
not  having  future  plans  with  English,  and  having  no  culture  on  using  technology  in  school.  Similarly,  Zhao  et  al. 
(2002)  proposed  some  explanations  such  as,  the  incompatibility  between  technology  and  the  current  culture  of 
schooling,  natural  unreliability  of  technology,  ill-preparedness  of  teachers,  poor  quality  of  educational  software,  the 
predominance  of  conservative  pedagogy,  and  the  power  of  standardized  assessment.  Likewise,  about  the  importance 
of  school  culture,  Zhao  et  al.  (2002)  emphasized  that  “[tjeachers  need  to  look  carefully,  not  only  within  themselves 
but  also  at  their  technological  and  social  environments  before  they  begin  to  implement  innovative  uses  of 
technology  in  their  own  classrooms  and  teaching”  (p.5 1 1).  Some  of  the  stated  reasons  could  be  accepted  as  general 
problems  of  Turkish  education  system.  For  instance,  Cakiroglu  and  Cakiroglu  (2003)  pointed  out  some  of  the 
problems  as  inadequacy  of  “professional  skills  and  knowledge  to  cope  with  the  educational  goals  of  today’s  society, 
low  salary,  low  status,  heavy  demands  upon  time,  heavy  workload,  lack  of  opportunities  to  improve  professional 
knowledge  and  effective  performance  and,  finally,  lack  of  job  security”  (p.257). 

Using  appropriate  assessment  strategies  allows  teachers  to  look  for  evidence  of  deeper  understanding, 
synthesis,  statements  of  relationships,  and  generalization  of  ideas  to  new  domains  (Dwyer,  1994).  In  the  study,  it 
seems  that  there  are  differences  between  private  and  public  high  school  English  teachers  on  NETS-T  “Evaluation 
and  Assessment”  indicators.  For  instance,  private  high  school  English  teachers  have  quite  much  knowledge  about 
NETS-T  “Evaluation  and  Assessment”  indicators  while  public  high  school  English  teachers  have  limited  knowledge 
on  them.  Using  technologies  in  their  lessons  in  private  high  schools  may  be  a cause  of  these  differences.  Another 
reasons  may  be  private  high  schools  have  more  technology  facilities  than  public  high  schools. 

Technology  helps  teachers  in  breaking  out  of  their  traditional  isolation,  communicating  with  their  peers 
(e.g.  through  online  forums)  and  outside  content  experts  about  the  instructional  content  and  pedagogical  issues,  and 
communicating  with  parents  about  their  expectations,  activities,  assignments,  and  student  progress  (Glennan  & 
Melrned,  1996).  However,  there  were  big  differences  on  the  NETS-T  “Productivity  and  Professional  Practice” 
indicators  between  private  and  public  high  school  English  teachers.  Private  high  school  English  teachers  had  far 
more  knowledge  on  the  NETS-T  “Productivity  and  Professional  Practice”  indicators  than  public  high  school  English 
teachers.  Private  high  school  English  teachers  were  using  technologies  in  their  teaching  more  than  public  high 
school  English  teachers.  Being  effective  in  their  teaching  and  holding  the  job  in  private  schools  may  require  teachers 
to  develop  themselves  rapidly  and  continuously.  Moreover,  private  high  school  administrators  may  be  demanding 
teachers  following  and  applying  technological  developments  to  their  teaching.  In  addition,  some  of  them  were  also 
providing  training  opportunities  for  their  previous  and  new  teachers.  The  findings  showed  that  public  high  school 
English  teachers  had  more  teaching  hours  in  a week  than  private  high  school  English  teachers  and  this  could  be 
another  reason  for  public  high  school  teachers'  not  sparing  time  for  their  own  professional  development. 

The  findings  of  the  study  also  showed  that,  only  nine  out  of  17  teachers  stated  formal  education  contributed 
to  their  professional  development  in  integration  of  technologies.  Even  though  the  findings  of  this  cannot  be 
generalized,  more  studies  should  be  conducted  to  examine  the  effectiveness  of  technology  integration  courses  in 
teacher  education  curricula  in  different  subject  areas  and  levels,  and  those  courses  need  to  be  in  line  with  the  subject 
area  teaching  rather  than  being  isolated. 

Teachers  should  consider  various  factors  related  with  the  social,  ethical,  legal,  and  human  issues.  For 
example,  Swenson  et  al.  (2005)  stated  that  instruction  and  homework  assignments  must  be  suitable  to  the  students’ 
technological  access  and  expertise,  so  that  each  student,  regardless  of  gender,  economic,  social,  ethnic,  or  linguistic 
backgrounds,  could  have  equal  learning  opportunities.  However,  in  this  study,  it  can  be  said  that  on  the  three  NETS- 
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T’s  “Social,  Ethical,  Legal,  and  Human  Issues”  indicators,  most  of  the  public  high  school  English  teachers  have 
inadequacy  of  knowledge.  However,  most  of  the  private  high  school  English  teachers  have  lack  of  knowledge  on 
only  one  indicator.  Similarly,  Akbaba-Altun  (2004)  conducted  a study  in  Turkey  and  found  that  although  school 
principals  should  know  how  to  deal  with  ethical  issues  emerging  with  IT  classrooms;  neither  school  principals  nor 
computer  coordinators  mentioned  any  expected  roles  about  ethics. 

To  conclude  as  Baran  (2007)  pointed  out  “history  of  teacher  education  in  Turkey  and  new  changes  in 
educational  system  showed  that  teacher  education  needed  more  technology  based  solutions”  (p.26).  The  teachers 
also  should  be  provided  with  inservice  training,  appropriate  work  load,  appropriate  technological  infrastructure  in 
schools,  and  technical  and  material  support  to  have  them  integrate  the  current  technology  in  their  teaching. 
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Examining  the  Foundations  of  Educational  Technology  Course  in  2009 

Jeremy  Tutty,  Florence  Martin  and  Zachary  McLaren 

Introduction 

The  introductory  course  or  course  sequence  is  common  among  educational  technology  programs  as  the 
means  to  provide  students  general  knowledge  of  instructional  design  and  technology.  Often,  this  course  or  courses 
provides  entry-level  students  with  their  first  exposure  to  both  historical  and  recent  developments  in  the  field  ( Klein, 
et  al,  2002;  Reiser  & Dempsey,  2007).  One  such  recent  development  is  the  2007  revision  of  the  Association  for 
Educational  Communications  and  Technology  ( AECT)  definition  of  educational  technology.  The  study  and  ethical 
practice  of  facilitating  learning  and  improving  performance  by  creating,  using,  and  managing  appropriate 
technological  processes  and  resources  (Januszewski  & Molenda,  2007  ). 

In  2002,  a needs  assessment  was  conducted  by  researchers  from  Arizona  State  University  to  identify 
optimal  instructional  content  for  a foundations  course  in  Educational  Technology  (Klein,  et  al,  2002).  Among  their 
findings,  they  concluded  that  introductory  courses  should  provide  more  opportunity  to  learn  about  contemporary 
issues  (Klein,  et  al,  2002).  Seven  years  later,  we  repeatedly  encounter  evidence  of  the  field  being  influenced  by 
growth  and  change  in  both  theoretical  and  technological  advances.  According  to  Januszewski  and  Molenda  (2007), 
“These  theoretical  shifts  have  changed  the  orientation  of  the  field  dramatically,  from  a field  driven  by  the  design  of 
instruction  to  be  delivered  in  a variety  of  formats  to  a field  which  seeks  to  create  learning  environments  in  which 
learners  can  explore  often  assisted  by  electronic  support  systems  in  order  to  arrive  at  meaningful  understanding.”  (p. 
2) 

In  light  of  these  recent  advances,  the  purpose  of  this  study  is  to  follow  the  admonition  of  Klein,  et  al  (2002) 
to  collect  additional  data  to  determine  the  optimal  instructional  content  and  delivery  method  for  the  foundations  of 
educational  technology  course.  In  this  current  needs  assessment  the  following  questions  will  be  addressed. 

1)  What  is  the  optimal  instructional  content  for  a foundations  course  in  Educational  Technology? 

2)  What  is  the  optimal  delivery  method  for  a foundations  course  in  Educational  technology? 

3)  What  feelings  do  respondents  have  about  the  new  technologies  available  for  the  delivery  of  a foundations  course? 

Method 

Phase  I of  the  study  - Syllabi  Analysis 

Faculty  who  prepare  instructional  design  and  technology  students  have  to  be  competent  practitioners, 
prepared  to  update  curriculum  based  on  the  changing  nature  of  the  field.  This  study  investigated  the  extent  to  which 
the  foundations  course  should  remain  a static  set  of  topics,  or  evolve  over  time.  This  study  follows  the  design  of  a 
similar  study  examining  research  courses  in  educational  technology  programs  (Klein,  Martin,  Tutty,  & Su,  2005).  In 
the  first  phase,  syllabi  from  a number  of  universities  and  the  topics  from  the  2002  needs  assessment  were  analyzed 
for  inclusion  in  a survey  instrument. 

Introductory  course  syllabi  were  collected  from  several  leading  educational  technology  programs  to 
determine  what  content  is  currently  being  taught  to  graduate  students  in  the  field.  Our  sample  included  eight 
programs:  Arizona  State  University,  Florida  State  University,  University  of  Georgia,  Indiana  University,  Purdue 
University,  San  Diego  State  University,  Syracuse  University,  and  Wayne  State  University.  The  syllabi  were  either 
downloaded  from  course  websites,  or  requested  from  the  instructors  teaching  the  course.  A content  analysis  of  the 
introductory  course  syllabi  was  conducted.  Each  syllabus  was  examined  for  course  title,  objectives,  instructional 
activities,  and  topics  covered. 

Phase  II  of  the  study  - Sur\’ey 

A survey  instrument  was  then  developed  to  address  issues  related  to  the  content,  use  of  technologies,  and 
delivery  method  of  introductory  courses  in  educational  technology.  Fifteen  course  topics  that  were  addressed  in  two 
or  more  programs  as  indicated  by  the  course  syllabi  were  identified  and  included  on  the  survey.  In  addition,  five 
topics  identified  by  Klein,  et  al.  (2002)  that  did  not  appear  on  two  or  more  of  the  syllabi  were  included.  A number  of 
open  ended  questions  were  added  to  the  survey. 

This  study  focused  on  the  optimal  instructional  content,  delivery  method,  and  emergent  technologies  for  a 
foundations  course  in  Instructional  Technology.  A survey  was  created  and  disseminated  to  various  higher 
educational  listserv  on  in  Fall  2009. 
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The  survey  consists  of  eight  total  questions;  five  required  questions  and  three  short  answer  questions  which 
were  not  required  for  survey  completion.  The  survey  measured  the  optimal  content  delivery  method,  and  emergent 
technologies  for  an  Instructional  Technology  foundations  course.  The  questions  were  grouped  into  four  categories: 
personal  background  information,  important  topics  in  an  Instructional  Technology  foundations  course,  delivery 
methods  in  an  Instructional  Technology  foundations  course,  and  contributions  of  emergent  technologies  to  an 
Instructional  Technology  foundations  course. 

The  survey  created  using  Select  Survey,  had  the  corresponding  link  distributed  through  email  to  the 
following  institutions  with  higher  educational  Instructional  Technology  listservs:  University  of  North  Carolina 
Wilmington,  University  of  South  Alabama,  Arizona  State  University,  University  of  North  Dakota,  Syracuse 
University,  University  of  Winnipeg,  and  Indiana  University. 


Associated  Universities  of  Survey 
Participants 

■ University  of  North  Carolina 
Wilmington 

■ Univeristy  South  Alabama 

■ Arizona  State  University 

■ University  of  North  Dakota 

■ Syracuse  University 

■ Indiana  University 


Figure  1.  Associated  Universities  of  Survey  Participants 

Participants 

Fifty-one  individuals  took  the  survey.  Nineteen  participants  (37%)  were  Instructional  Technology  master 
students,  11  participants  (22%)  were  Instructional  Technology  doctoral  students,  8 participants  (16%)  did  not  fit 
within  the  given  descriptions  and  are  listed  as  other,  5 Participants  (10%)  were  Instructional  Technology  practioners 
in  the  K-12  range,  3 participants  (6%)  were  Instructional  Technology  practioners  in  the  Business  and  Industry 
sector,  2 participants  (4%)  were  Instructional  Technology  Faculty,  another  2 participants  (4%)  were  Instructional 
Technology  practioners  in  Higher  Education  and  1 participant  (2%)  was  an  Instructional  Technology  practioner  for 
the  government. 

Results 

The  results  of  the  survey  are  reported  in  Table  1.  The  overall  mean  of  all  items  was  (M=4.23)  on  a scale  of 
1 - Very  Unimportant  to  5 Very  Important.  Ten  topic  items  were  rated  above  the  mean  and  were  considered 
important  to  very  important  for  Instructional  Technology  foundation  courses.  Among  these  ten  topics  the  five  most 
important  were:  Systematic  Instructional  Design  (M=  4.75),  Instructional  Design  Models  (M=  4.75),  Learning 
Theories  (M=  4.61),  Instructional  Theories  (M=  4.59),  and  Instructional  Media  Design  (M=  4.45).  Though  the 
remaining  10  topics  were  below  the  overall  mean  of  (M=  4.23),  they  were  all  rated  as  important  with  their  means 
ranging  from  (M=  4.22)  to  (M=  3.75).  History  of  educational  technology  and  professional  competencies  (IBSTPI) 
were  rated  the  lowest  at  (M=3.75). 
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Instructional  Design  Foundations  Survey 


Table  1 


Topics  in  an  Instructional  Technology  Foundations  Course 


Topics  in  Foundation  Instructional  Technology 

(5=  Very  Important,  4=  Important,  3=  Neutral,  2=  Unimportant,  1 = Very  Unimportant 

Average 

Response 

1 

Instructional  Design  Models 

4.75 

2 

Systematic  Instructional  Design 

4.75 

3 

Learning  Theories 

4.61 

4 

Instructional  Theories 

4.59 

5 

Instructional  Media  Design 

4.45 

6 

Needs  Assessment 

4.43 

7 

Evaluation 

4.39 

8 

Definitions  of  Educational  Technology 

4.37 

9 

Trends  in  Educational  Technology 

4.33 

10 

Gagne's  Events  of  Instruction 

4.29 

11 

Using  Professional  Resources 

4.22 

12 

Human  Performance  Technology 

4.06 

13 

Innovation  & Change 

4.06 

14 

Technology  Integration 

4.02 

15 

Media  Selection 

4.00 

16 

Adoption  & Diffusion 

3.98 

17 

Distance  Education 

3.96 

18 

Technology  Standards  (ISTE) 

3.84 

19 

History  of  Educational  Technology 

3.75 

20 

Professional  Competencies  (IBSTPI) 

3.75 

The  results  for  delivery  methods  are  reported  in  Table  2.  Course  Websites  were  rated  the  most  appropriate 
delivery  method  (M=  4.47)  on  a scale  of  1-  Very  Inappropriate  to  5 Very  Appropriate.  Learning  Management 
Systems  (Eg.  Blackboard.  Moodle)  and  Virtual  Classrooms  (Eg.  Wimba,  Elluminate,  Connect)  were  rated  the  next 
most  appropriate  delivery  method  with  means  above  (M=3.35).  Slightly  lower  in  the  middle  of  the  results  were 
Video  Podcasts  (M=  4.12),  Audio  Podcasts  (M=  3.84),  Wikis  (M=  3.80),  and  Blogs  (M=  3.67)  all  deemed 
appropriate  delivery  methods.  Rounding  out  the  bottom  of  the  results  were  Social  Networking  Sites  (Eg.  Ning)  (M= 
3.53),  Mobile  Devices  (Eg.  iphone.  Blackberry)  (M=  3.31),  and  Virtual  Environments  (Eg.  Second  Life)  (M=  3.18) 
all  being  deemed  as  slightly  appropriate  delivery  methods  for  blended  online  foundations  course  in  Instructional 
Technology. 

Table  2 


Delivery  methods  for  a blended  or  an  online  foundations  course  in  Instructional  technology 


Delivery  methods  for  a blended  or  an  online  foundations  course  in  Instructional  Technology 

(5=  Very  Appropriate,  4=  Appropriate,  3=  Neutral,  2=  Inappropriate,  1=  Very  Inappropriate) 

Average 

Response 

1 

Course  Websites 

4.47 

2 

Learning  Management  Systems  (Eg.  Blackboard,  Moodle) 

4.41 

3 

Virtual  Classrooms  (Eg.  Wimba,  Elluminate,  Connect) 

4.39 

4 

Video  Podcasts 

4.12 

5 

Audio  Podcasts 

3.84 

6 

Wikis 

3.80 

7 

Blogs 

3.67 

8 

Social  Networking  Sites  (Eg.  Ning) 

3.53 

9 

Mobile  Devices  (Eg.  iphone.  Blackberry) 

3.31 

10 

Virtual  Environments  (Eg.  Second  Life) 

3.18 

The  results  for  the  contribution  of  emergent  technologies  within  a blended  online  foundations  course  in 
Instructional  Technology  are  reported  in  Table  3.  Contributions  were  rated  on  a scale  of  1 - Not  at  all  Significant  to 
5 - Very  Significant.  Among  the  ten  items  in  this  category,  the  item.  Help  students  stay  current  with  technology 
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(M=  4.41)  was  reported  to  be  the  most  significant  contribution  of  emergent  technologies  to  Instructional  Technology 
Foundation  courses.  The  items:  Provides  student  with  the  flexibility  in  time.  Prepare  students  for  the  job  market. 
Provides  student  with  the  flexibility  in  space,  and  Encourage  student-student  interaction  were  rated  above  (M=3.95) 
and  are  also  considered  significant  reasons  for  the  use  of  emergent  technologies  in  Instructional  Technology 
foundation  courses.  The  remaining  5 items  rated  below  (m=3.95)  which  were  still  considered  significant  were: 
Encourage  faculty-student  interaction  (M=3.90),  Provides  students  a sense  of  community  (M=3.67),  It  enhances 
student  effectiveness  (M=3.67),  Help  students  succeed  in  the  course  (M=3.65),  and  It  increases  student  performance 
(M=3.49).  There  were  no  ratings  below  (M=3.0)  which  could  have  been  considered  insignificant. 

Table  3. 


Contributions  of  Emergent  Technologies 


Emergent  Technologies 

(5=  Very  Significant,  4=  Significant,  3=  Neutral,  2=  Insignificantly,  1 = Not  at  all) 

Average 

Response 

1 

Help  students  stay  current  with  technology 

4.41 

2 

Provides  student  with  the  flexibility  in  time 

4.29 

3 

Prepare  students  for  the  job  market 

4.24 

4 

Provides  student  with  the  flexibility  in  space 

4.04 

5 

Encourage  student-student  interaction 

3.98 

6 

Encourage  faculty-student  interaction 

3.90 

7 

Provides  students  a sense  of  community 

3.67 

8 

It  enhances  student  effectiveness 

3.67 

9 

Help  students  succeed  in  the  course 

3.65 

10 

It  increases  student  performance 

3.49 

Twenty  of  the  51  respondents  completed  at  least  one  of  the  three  open-ended  survey  items.  When  asked 
what  additional  topics  or  trends  not  identified  in  the  survey  should  be  covered  in  the  foundations  course,  responses 
reflected  a desire  for  increased  practical  application  of  the  topics  indicated  in  the  survey.  When  asked  about 
preference  for  other  methods  of  delivering  the  foundations  course,  8 of  14  (57%)  respondents  indicated  they  had  no 
preference  for  additional  delivery  methods,  4 of  14  (29%)  respondents  indicated  a preference  for  social  networking 
tools,  and  the  remaining  two  respondents  complained  about  course  management  systems.  When  asked  to  identify 
additional  benefits  of  using  emergent  delivery  methods,  13  of  20  (65%)  respondents  identified  increased  access  and 
engagement.  The  remaining  responses  did  not  address  the  item. 

Discussion 

According  to  Januszewski  and  Molenda  (2007),  “Theoretical  shifts  [in  instructional  technology]  have 
changed  the  orientation  of  the  field  dramatically,  from  a field  driven  by  the  design  of  instruction  to  be  delivered  in  a 
variety  of  formats  to  a field  which  seeks  to  create  learning  environments  in  which  learners  can  explore  often  assisted 
by  electronic  support  systems  in  order  to  arrive  at  meaningful  understanding.”  (p.  2). 

The  results  of  this  study  seem  to  fall  short  of  fully  endorsing  the  assertions  by  Januszewski  and  Molenda. 
Our  findings  certainly  do  place  a high  value  on  the  communicative  and  collaborative  benefits  of  electronic  support 
systems.  Furthermore,  respondents  recognized  the  importance  of  fluency  with  technological  tools,  rating  items 
related  to  staying  current  with  technology  consistently  high.  Delivery  mode  appears  to  be  driven  by  convenience  not 
a particular  advantage  of  one  mode  or  another. 

However,  results  also  reinforce  the  need  to  maintain  sound  instructional  design  practices  in  the  face  of 
emergent  technological  demands.  It  maybe  argued  to  the  extent  that  such  practices  remain  the  driving  force  in  the 
field.  Respondents  placed  the  greatest  value  on  the  topics  of  Systematic  Instructional  Design,  Instructional  Design 
Models,  Learning  Theories,  and  Instructional  Theories;  not  social  networking  or  virtual  environments.  It  would 
seem  that  even  in  this  current  age  of  unparalled  technological  advance,  the  historical  foundations  of  the  field  still 
belong  in  the  foundations  course. 

It  may  be  somewhat  presumptive  to  challenge  to  driving  force  of  the  field  based  upon  the  content  of  a 
foundations  course,  but  As  one  respondent  indicated,  “It  is  not  the  technology  that  does  all  these  things.  It  is  the 
instruction  behind  how  the  technology  is  used  to  facilitate  and  support  learning  and  instruction.  All  of  these  things 
can  be  enhanced  or  distracted  by  technology  used.” 
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Abstract:  To  explore  potential  applications  of  Second  Life  (SL)  for  language  learning  and  instruction,  researchers 
from  an  American  university  and  a Chinese  university  took  an  evaluation  research  approach  to  identify  appropriate 
ways  to  integrate  SL  into  an  EFL  (English  as  a Foreign  Language)  program  in  China.  The  paper  reports  the  research 
efforts  in  the  two  pilot  EFL  Programs  implemented  online  in  SL  (spring  2008  and  spring  2009),  the  program 
evaluation  framework.  It  also  makes  suggestions  for  implementing  international  research  collaboration  for  similar 
EFL  programs  in  SL  based  on  preliminary  analysis  of  the  data. 

Key  words:  International  research  collaboration.  Second  Life,  design-based  research,  EFL  learning. 

Introduction 

Educational  researchers  and  practitioners  have  paid  considerable  attention  to  the  use  of  multiuser  virtual 
environments  (MUVEs)  in  recent  years  in  order  to  support  learning  across  curricula.  According  to  Nelson  and 
Ketelhut:  “Educational  MUVEs  have  emerged  in  recent  years  as  a form  of  socio-constructivist  and  situated 
cognition-based  educational  software”  (2007,  p.  269).  Among  MUVEs,  Second  Life  (SL)  is  one  of  the  most  popular 
environments,  with  over  18  million  registered  users  as  of  May  2008  (Second  Life  Economic  Statistics,  May  2008). 
This  paper  reports  a recent  endeavor  of  research  collaboration  between  an  American  university  and  a Chinese 
university  to  explore  various  facets  of  EFL  (English  as  a Foreign  Language)  learning  within  SL.  The  ultimate 
purpose  of  this  collaboration  is  to  find  appropriate  ways  to  integrate  SL  into  EFL  programs  in  China. 

The  exploration  and  research  used  in  this  project  were  informed  by  social  constructivist  principles  applied 
to  the  context  of  EFL  learning.  The  approach  used  to  investigate  related  issues  and  problems  for  practical  solutions 
is  evaluation  research  (Krathwohl,  1998;  McMillan  & Schumacher,  1997).  In  spring  2008  and  spring  2009,  two  pilot 
programs  of  the  EFL  program  in  SL  were  completed  online  with  participants  from  both  universities.  Our  efforts  to 
this  point  have  been  to  explore  (a)  how  the  affordances  of  SL  might  be  a useful  tool  that  can  mediate  EFL  learning, 
(b)  the  effects  of  the  EFL  Program  in  SL  on  Chinese  students’  oral  proficiency,  (c)  the  Chinese  students’  perceived 
technology  readiness  for  using  SL,  and  (4)  the  Chinese  students’  perceptions  of  SL  and  the  EFL  Program 
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implemented  in  SL.  The  paper  provides  background  information  on  the  research  collaboration,  the  methods  used  in 
the  EFL  program  conducted  with  SL  and  suggestions  for  implementing  the  similar  programs  for  international 
collaboration. 


The  Research  Project  Background 

The  research  collaboration  was  initiated  when  two  faculty  members  from  the  American  university  and  the 
Chinese  university  met  through  the  Summer  Fellowship  Program  of  the  Society  of  International  Chinese  in 
Educational  Technology  (SICET).  SICET  is  a nonpolitical,  non-profit  academic  organization  affiliated  with 
Association  for  Educational  Communications  and  Technology  (AECT).  The  mission  of  SICET  is  to  establish  and 
strengthen  academic  international  connections,  exchanges,  research,  and  studies  in  educational  technology  for 
teaching  and  learning,  as  well  as  to  promote  the  application  of  educational  technology  in  Chinese  education  (SICET, 
2009).  The  SICET  Summer  Fellowship  Program  offers  opportunities  for  SICET  members  to  visit  Chinese 
universities  and  to  share  their  research  findings  with  their  Chinese  counterparts  through  conference  presentations, 
guest  lectures,  and  seminars  at  the  host  institutes  in  China.  Nearly  ten  universities  and  institutes  in  China  have 
hosted  SICET  Summer  Fellowship  Program  events  as  of  May  2009. 

In  addition  to  the  common  research  interests  shared  by  these  two  researchers,  two  major  factors  informed 
this  research  collaboration.  First,  there  was  the  need  for  EFL  programs  in  China  to  find  appropriate  ways  to  integrate 
new  technologies  to  their  programs.  Second,  there  has  also  been  an  emerging  discussion  about  the  use  of  multi-user 
virtual  environments  (MUVEs)  as  tools  for  second  and  foreign  language  instruction  (Cooke-Plagwitz,  2008).  This 
phenomenon  may  have  evolved  from  two  language  research  areas:  Computer  Assisted  Language  Learning  (CALL) 
and  Computer  Mediated  Communication  (CMC)  (Lam,  2000;  Shei,  2005).  Yet,  little  is  known  about  how  MUVEs, 
Second  Life  (SL)  in  particular,  impact  the  ways  in  which  language  learners  improve  their  language  proficiency. 
Despite  its  great  potentials  to  augment  language  instruction,  very  little  research  has  been  conducted  on  the  use  of  SL 
in  the  context  of  learning  English  as  a Foreign  Language  (EFL)  - specifically  in  contexts  where  English  is  not 
spoken  in  everyday  life,  but  is  often  limited  to  the  classroom  (e.g.,  learning  English  in  China). 

The  EFL  Programs  in  Second  Life 

During  spring  2008  and  spring  2009,  two  pilot  EFL  programs  were  implemented  in  SL  with  participants 
from  both  universities.  The  American  university,  founded  in  1913,  is  the  Southeast's  leading  urban  research 
institution.  It  offers  52  degree  programs  with  250  fields  of  study  through  six  colleges  with  nearly  30,  000 
undergraduate  and  graduate  students.  The  Chinese  university,  founded  in  1984,  is  a key  provincial  university  in 
eastern  China.  It  is  a comprehensive  university  with  twenty  one  academic  colleges  and  schools  and  3 1 research 
institutes  and  centers.  It  offers  49  degree  programs  and  has  an  enrollment  of  nearly  29,000  undergraduate  and 
graduate  students. 
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The  first  EFL  Program  in  SL  (spring  2008) 


For  the  first  pilot  EFL  Program  in  SF  (spring  2008),  five  American  graduate  students  participated.  Three  of 
the  graduate  students  were  doctoral  students  in  the  Instructional  Design  and  Technology  program  and  the  other  two 
were  graduate  students  of  the  Applied  Linguistics  program.  The  American  participants  were  proficient  in  SL  and 
were  prepared  to  interact  with  their  Chinese  counterparts  in  SL  through  participation  in  a short  workshop. 

The  Chinese  participants  consisted  of  31  English  major  sophomores  from  the  English  Department  at 
Schools  of  Foreign  Languages.  The  program  in  SL  was  implemented  as  part  of  a two-hour  weekly  speaking  class  in 
the  English  Department.  The  instructor  of  the  course  designed  activities  centered  on  topics  which  included  the  topic 
of  globalization  as  a context  for  improving  students’  English  speaking  skills  through  reading  and  discussion.  A set 
of  SL  learning  activities  were  designed  by  researchers  as  part  of  guided  practice  as  well  as  a means  for  the  students 
to  interact  with  American  students. 

The  Chinese  participants  first  completed  a survey  designed  to  gauge  their  technology  readiness  for  using 
SL  for  EFL  learning.  Immediately  after  the  survey,  they  participated  in  a one -hour  workshop  designed  to  familiarize 
them  with  basic  navigational  and  communication  functions  of  SL.  After  the  workshop,  participants  interacted  with 
their  Chinese  peers  in  SL  to  both  familiarize  themselves  with  the  environment  and  to  prepare  a list  of  questions 
about  globalization  to  guide  the  interview  with  the  American  participants. 

Next,  the  Chinese  participants  were  divided  into  two  groups.  In  the  first  group,  sixteen  participants 
interacted  one-on-one  with  an  American  participant  to  complete  the  given  tasks  in  SL.  In  the  second  group,  fifteen 
Chinese  participants  formed  three  sub-groups  of  five  to  interact  with  other  American  participants.  All  of  the 
participants  (both  the  American  and  the  Chinese)  logged  into  SL  using  preconfigured  avatars.  The  avatars  had  been 
placed  in  predetermined  locations  within  SL.  Chinese  participants  and  their  American  counterparts  were  given  two 
language  tasks.  The  first  task  for  Chinese  participants  was  to  interview  an  American  student  about  his/her 
perspectives  on  globalization  with  the  goal  of  writing  an  article  for  their  university  newsletter.  The  second  task  was 
for  the  American  participants  to  interview  the  Chinese  participants  about  university  student  life  in  China.  The  first 
task  was  designed  to  help  Chinese  students  practice  oral  skills  such  as  questioning  and  clarification  (or  information 
seeking).  The  second  task  was  designed  to  help  Chinese  participants  practice  answering  questions  in  English.  The 
participants’  language  performances  were  recorded  using  screen  recording  software  in  the  language  labs  at  both 
universities.  The  Chinese  participants  also  completed  post  program  survey  followed  by  a focus  group  interviews. 

The  second  EFL  Program  in  SL  (Spring  2009) 

The  second  EFL  Program  in  SL  was  implemented  in  spring  2009.  The  participants  from  the  American 
university  were  20  undergraduate  students  who  were  taking  the  course  IT  2010:  Computer  Skills  for  Information 
Age.  Their  participation  in  the  study  was  part  of  their  course  work  requirement.  The  American  students  had  a two 
hour  workshop  on  SL  before  they  engaged  in  learning  activities  with  their  Chinese  counterparts  in  SL. 

61  full  time  English  majored  sophomores  at  the  Chinese  university  completed  a study  survey  of  technology 
readiness  for  using  SL  for  EFL  learning.  From  the  61  Chinese  students,  20  were  selected  to  participate  in  the  EFL 
Program  in  SL  including  seven  female  students  (35%)  and  13  male  students  (65%).  Their  ages  ranged  from  19  to  23 
with  an  average  of  21.  The  participants  were  selected  on  a voluntary  basis  and  according  to  their  comprehensive 
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evaluation  scores  of  the  previous  academic  semester  (fall  2008).  The  comprehensive  evaluation  consisted  of  the 
evaluation  on  students’  listening  comprehension,  speaking  proficiency,  reading  and  writing  abilities.  The  evaluation 
was  done  by  professors  at  the  university  on  a semester  basis  to  assess  students’  academic  performance. 

The  second  EFL  Program  in  SL  lasted  five  weeks,  from  March  24  to  April  26,  2009.  The  program  provided 
the  participants  with  opportunities  to  interact  with  American  students  synchronously  online  in  SL.  The  learning 
tasks  for  each  week  consisted  of:  (1)  one  hour  of  learning  activities  in  SL,  (2)  blog  writing  on  the  learning 
experiences  in  SL,  and  (3)  language  preparation  for  learning  activities  in  SL  for  the  next  week.  The  learning 
activities  in  SL  included  a SL  workshop  in  a lecture  format,  virtual  tours  in  groups  and  one-on-one  with  SL  partners, 
small  group  discussions,  one-on-one  interview  with  the  American  partners,  and  individual  presentations.  A Google 
Sites  website  was  created  to  give  detailed  instructions  on  how  to  complete  the  tasks  for  each  week.  Following  given 
instructions  on  the  web  site,  the  Chinese  participants  completed  their  tasks  at  an  Internet  Center  on  the  Chinese 
university  campus.  Two  Chinese  research  assistants  were  present  at  the  site  providing  necessary  technical  support. 
Figure  1 is  the  screenshots  of  the  participants  completing  different  learning  tasks  in  SL  and  Table  1 summarizes  the 
tasks  for  the  EFL  Program  participants  at  the  Chinese  university. 


Virtual  Tour  Presentation 


Figure  1:  Different  Learning  Tasks  in  SL 
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Program  Evaluation 


For  both  the  formative  and  summative  evaluation  of  the  2008  and  2009  pilot  EFL  Programs,  qualitative  and 
quantitative  analyses  (Creswell,  2008)  were  used  in  order  to  improve  the  EFL  Program  in  SL.  Our  evaluation 
framework  is  detailed  in  Table  1 below. 


Table  1:  Evaluation  Framework  for  the  Pilot  EFL  Programs  in  Second  Life 


Major  Evaluation  Question:  What  should  we  do  to  improve  the  EFL  Program  in  SL? 

Evaluation  Contents  and  Questions 

Methods  & Data  Sources 

Technology  readiness: 

• Are  computers  in  the  language  lab  ready  to  use  SL  for 
language  learning? 

0 SL  use  with  American  students 
0 Video  recording  and  data  saving 

• Site  testing  at  both  the  American  university 
and  the  Chinese  university 

Instruments: 

• Are  the  surveys  clear  to  the  Chinese  students? 

• Are  the  instructions  on  learning  activities  appropriate 
for  the  Chinese  students  in  terms  of  language  level  and 
activity  contents? 

• Subject  Matter  Expert  reviews 

• Target  student  tests  and  comments 

The  Chinese  students’  technology  readiness  for  using 
SL  for  learning: 

• Are  the  Chinese  students  technologically  ready  to  use 
SL  for  EFL  learning? 

• What  are  the  self-perceived  readiness  of  the  Chinese 
students  for  perform  language  tasks  in  SL? 

• Pre-program  and  post  program  surveys 

• Interviews 

• Blog  posting 

• Site  observation 

The  Chinese  students’  attitude: 

• What  are  the  Chinese  students’  attitudes  toward  using 
SL  for  EFL  learning? 

• Post  program  Survey 

• Interviews 

• Blog  posting 

Effects  on  learning: 

• What  are  the  Chinese  EFL  students'  language 
performances  in  SL? 

• What  are  major  benefits  of  SL  perceived  by  the 
Chinese  students  for  language  learning? 

• What  are  major  barriers  in  using  SL  for  language 

• Pre  and  post  oral  tests 

• Assessment  rubrics  for  oral  tests 

O Accuracy 
O Fluency 

O Language  complexity 
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learning  perceived  by  the  Chinese  students? 

• What  are  the  suggestions  from  the  Chinese  students  for 
program  improvement? 

0 Language  appropriateness 

• Recorded  language  performances  in  SL 

• Interviews 

• Blog  posting 

The  American  students  comments: 

• What  comments  do  American  students  made  on  the 
Chinese  students’  language  performances  in  SL? 

• Post  program  written  report 

The  second  EFL  Program  generated  a vast  amount  of  data  that  are  currently  under  analysis  and  will  be 
summarized  in  a written  report.  These  data  include  94  recorded  oral  tests,  99  video  clips  of  students’  language 
performance  in  SL,  18  recorded  interviews,  77  blog  postings,  as  well  as  over  77  pre  and  post  program  surveys. 

Recommendations 

Evaluation  is  a systemic  approach  to  program  improvement,  and  program  improvement  should  lead  to 
improved  student  learning.  Evaluation  should  lead  to  recommendations  intended  to  optimize  the  program  in  relation 
to  its  intended  purposes  (Fitzpatrick,  Sanders,  & Worthen,  2004).  According  to  the  program  evaluation  results,  we 
recommend  the  following  for  designing  and  implementation  of  similar  future  research  program. 

1.  Create  multiple  channels  for  online  communications  among  research  team  members.  Due  to  the  nature  of 
the  research  collaboration,  the  exchanges  of  ideas  and  documents  are  frequent  and  common.  The  research  team 
should  establish  and  use  a file  sharing  system  (e.g.  Google  docs.),  and  an  online  chat  medium  (e.g.  QQ,  Skype)  that 
includes  both  audio  and  video  chat  capability  for  international  communications. 

2.  Test  the  technology  at  the  site.  From  the  pilot  EFL  Programs,  we  came  to  see  both  successes  and  failures 
that  technology  may  bring  to  the  EFL  Programs  in  SL.  We  recommend  that  researchers  test  the  technology  prior  to 
the  start  of  each  EFL  program  activity  at  each  of  the  sites. 

3.  Involve  graduate  students  at  both  sides  of  the  international  border  in  the  collaboration.  Research 
collaboration  on  such  a scale  requires  a lot  of  detail-oriented  work.  Involving  graduate  students  on  both  sides  not 
only  makes  it  possible  to  collect  and  analyze  the  data  effectively  and  efficiently  but  also  help  the  graduate  students 
learn  and  grow  as  educational  researchers. 

4.  Use  a wide  array  of  learning  tasks.  We  suggest  that  the  future  EFL  programs  use  a wide  array  of  tasks  that 
can  elicit  structured  interactions  (e.g.,  interviewing  native  speakers)  as  well  as  semi-structured  or  improvised 
interactions  (e.g.  virtual  field  trips  to  historically  significant  place,  visits  to  virtual  museums,  organizing  and 
participating  in  virtual  conferences,  designing  and  constructing  cultural  centers,  and  creating  virtual  art  shows,  etc.). 
Meaningful  interactions  in  SL  would  not  only  equip  the  EFL  students  with  knowledge  about  English  speaking 
cultures  but  also  provide  them  with  appropriate  ways  of  expressing  themselves  under  certain  social  circumstances. 

5.  Be  careful  in  the  site  selection.  The  selection  of  virtual  environments  needs  to  be  carefully  considered. 
There  are  many  objects  in  SL  that  could  interfere  and  disrupt  students  when  attempting  to  complete  given  tasks.  For 
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example,  it  may  not  be  appropriate  to  select  a seashore  site  in  SL  where,  among  other  things,  several  dolphins 
regularly  jump  off  the  sea  when  the  students  need  to  concentrate  on  their  discussion  that  are  not  related  to  dolphins. 
6.  Integrate  multiple  learning  formats  into  the  EFL  Program  in  SL.  The  ultimate  purpose  of  the  research 

collaboration  is  to  help  the  Chinese  students  with  EFL  learning  with  a focus  on  speaking.  However,  EFL  learning 
should  not  be  just  limited  to  speaking  in  SL.  Creating  other  channels  of  communication  (such  as  email,  blog  writing, 
Facebook,  and  etc.)  with  American  students  and  engaging  students  in  writing  are  also  beneficial  to  EFL  learning. 

The  rapid  evolution  of  digital  media  has  afforded  us  the  opportunity  to  assimilate  new  technologies  while 
exploring  new  pedagogies  in  EFL  language  learning.  We  believe  that  our  work  described  herein  has  barely  touched 
upon  this  potential  and  we  are  excited  about  what  the  future  may  bring.  Synchronous  and  asynchronous  information 
and  communications  technology  (ICT)  provides  opportunities  for  research  collaboration  across  international 
borders.  The  ability  to  communicate  at  large  distances  provides  both  the  opportunity  for  the  sharing  of  culture  and 
language,  and  increased  opportunity  for  collaborative  research  projects.  While  there  are  many  challenges  associated 
with  collaborative  projects  across  international  borders,  there  are  tremendous  opportunities  for  both  the  participants 
and  the  researchers  on  both  sides  of  border.  The  potential  for  ICT  and  MUVE  to  improve  communication  between 
students  from  different  languages  and  cultures  is  an  area  in  which  further  research  is  needed. 
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The  Teachers  Learning  in  Networked  Communities  (TLINC)  project  was  developed  by  the  National 
Commission  on  Teaching  and  America’s  Future  (NCTAF)  with  the  goals  of  improved  teacher  retention  and 
accelerated  teaching  proficiency  for  new  teachers.  The  goal  of  the  TLINC  project  is  to  reduce  the  professional  and 
personal  isolation  experienced  by  preservice  and  early-career  teachers.  NCTAF  has  partnered  with  three  universities 
(and  their  affiliated  school  districts)  in  the  United  States  to  create  and  sustain  online  communities  of  teachers  via  the 
TLINC  project.  The  three  universities  are  the  University  of  Memphis,  the  University  of  Colorado  Denver,  and  the 
University  of  Washington. 

Early-career  teacher  attrition  is  a significant  and  costly  problem  facing  America’s  schools.  One  reason  for 
teacher  attrition  is  the  isolation  experienced  by  educators  beginning  their  careers.  In  a teacher  education  program 
(TEP),  students  are  prepared  apart  from  the  schools  where  they  will  serve.  They  are  then  placed  as  stand-alone 
teachers  in  self-contained  classrooms  (NCTAF,  2006).  When  a teacher  candidate  leaves  the  TEP,  he  or  she  leaves 
behind  a support  system  of  teacher  educators  and  peers.  TLINC  seeks  to  address  this  isolation  by  focusing  on  three 
areas  of  need  identified  by  novice  teachers:  access  to  high-quality  teaching  resources,  frequent  access  to  experts 
(mentors  and  coaches),  and  ongoing  peer  support  (Silberstein,  Martindale,  & Young,  2007). 

These  three  areas  of  need  can  be  met  through  the  implementation  of  collaborative  mentoring  groups.  Face- 
to-face  groups  have  frequently  been  used  to  meet  these  needs.  However  with  the  many  responsibilities  teachers  have 
to  fulfill  and  the  sprawling  geography  of  some  school  districts,  face-to-face  meetings  are  not  always  feasible.  In 
these  situations,  online  meetings  are  a viable  substitute.  They  can  enhance  the  benefits  of  collaborative  groups  by 
overcoming  the  limitations  of  fixed  locations  and  schedules.. 

The  TLINC  project  is  working  to  implement  online  collaborative  groups  by  using  the  Tapped  In  online 
platform  (www.tappedin.org)  with  each  of  its  three  partner  sites.  The  TLINC  project  seeks  to  achieve  the  five 
following  outcomes  through  these  networked  communities:  1)  improved  teacher  retention;  2)  accelerated 
proficiency  for  new  teachers;  3)  opportunities  for  all  teachers,  administrators,  and  university  faculty  to  become 
engaged  in  a learning  community  that  continues  to  evolve;  4)  establishment  of  partnership  capacity-building 
structures  and  processes  that  assure  sustainability;  5)  identification  of  the  elements  of  TLINC  that  are  the  source  of 
its  power,  to  identify  the  essentials  for  replication  and  scaling. 

At  the  University  of  Memphis,  students  are  assigned  to  small  groups  and  a mentor  during  their  semester  of 
student  teaching.  The  goal  of  the  Memphis  TLINC  Project  is  that  all  student  teachers  would  form  online 
communities  of  practice.  The  purpose  of  forming  these  small  groups  is  to  allow  student  teachers  to  become  a part  of 
a group  that  will  lessen  their  feelings  of  isolation  and  improve  their  professional  expertise.  The  TLINC  project  is 
designed  to  provide  early-career  teachers  with  skills,  tools,  and  support. 

Review  of  Literature 

Beginning  teacher  attrition  is  a serious  problem  facing  America’s  schools.  Up  to  1/3  of  all  new  teachers 
leave  the  profession  within  three  years  of  teaching,  and  46%  of  teachers  leave  the  profession  after  five  years  of 
teaching.  The  number  of  teachers  leaving  the  profession  has  been  steadily  rising  since  1991  and  has  surpassed  the 
number  of  new  entrants  since  1992  (Fulton,  Yoon,  & Lee,  2005;  Silberstein,  Martindale,  & Young,  2007). 
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Teacher  attrition  is  not  isolated  to  any  specific  region.  The  high  rate  of  teacher  turnover  can  be  observed 
across  all  sectors  of  education.  The  effect  of  this  mass  departure  from  teaching  means  that  in  the  average  school, 
almost  half  of  the  teachers  have  less  than  three  years  of  experience  (Fulton,  Yoon,  & Lee,  2005;  Silberstein, 
Martindale,  & Young,  2007). 

One  cause  of  beginning  teacher  attrition  is  the  reported  isolation  felt  by  new  teachers.  New  teachers  are 
placed  in  stand-alone  classrooms  and  work  independently  from  their  peers  (Fulton,  et  al.,  2005).  Teacher  induction 
programs  and  professional  development  are  being  used  to  lessen  this  feeling  of  isolation  in  an  attempt  to  reduce 
teacher  attrition.  There  has  been  much  research  on  teacher  induction  and  professional  development  programs  for 
teachers  (see  e.g.,  Alavarado,  2006;  Corcoran,  Shields,  & Zucker,  1998;  Darling-Hammond,  Lieberman,  & 
Mclaughlin,  1995;  Fulton,  et  al.,  2005;  Schlager  & Fusco,  2004;  Schlager,  Fusco,  & Schank,  1998;  Smith  & 
lngersoll,  2004).  Two  possible  solutions  to  the  problem  of  attrition  for  early  career  educators  are  explored:  teacher 
induction  programs  and  professional  development. 

Teacher  Induction 

The  number  of  teacher  induction  programs  has  been  on  the  rise  in  recent  years.  These  programs  seek  to 
offer  support  and  guidance  for  beginning  teachers  in  their  new  profession.  In  this  study,  we  will  use  the  term 
induction  as  described  by  Smith  and  lngersoll  (2004).  Induction  programs  are  not  additional  teacher  training,  but 
rather  designed  for  educators  who  have  completed  their  teacher  training.  “These  programs  are  often  conceived  as  a 
bridge,  enabling  the  ‘student  of  teaching'  to  become  a ‘teacher  of  students’”  (p.  683).  Researchers  have  shown  the 
importance  of  beginning  this  process  during  a potential  teacher’s  preservice  years  (Boreen  & Niday,  2000;  Gutierrez 
& Basile,  2007).  We  recognize  the  relevance  of  starting  these  programs  during  preservice;  however,  to  stay  true  to 
the  description  set  forth  by  Smith  and  lngersoll,  we  will  only  examine  preservice  teachers  who  have  completed  all 
coursework  and  have  begun  the  clinical  experience  of  student  teaching. 

When  discussing  teacher  induction,  it  is  important  to  explore  the  effective  delivery  of  the  programs. 

Schlager,  Fusco,  Koch,  Crawford,  and  Philips  (2003)  give  three  components  of  programs  aiming  to  reduce  the 
attrition  rate  are:  (1)  access  to  high-quality  teaching  resources,  (2)  frequent  access  to  experts  (i.e.,  mentors  and 
coaches),  and  (3)  ongoing  peer  support. 

While  induction  programs  vary  in  their  depth  and  duration,  almost  all  involve  mentoring.  For  the  purposes  of 
this  study,  the  mentoring  component  of  teacher  induction  will  be  our  focus.  Again,  we  will  use  the  Smith  and 
lngersoll  (2004)  definition  of  mentoring,  “Mentoring  is  the  personal  guidance  provided,  usually  by  seasoned 
veterans,  to  beginning  teachers  in  schools”  (p.  683).  It  is  important  to  keep  in  mind  that  while  most  induction 
programs  involve  mentoring,  mentoring  is  not  the  only  component  and  the  two  are  not  synonymous  (Fulton,  et  al., 
2005). 

Some  school  systems  offer  mentoring  programs  to  all  teachers  new  to  a particular  school  and  others  offer 
them  only  to  those  teachers  who  are  new  to  the  profession  (Smith  & lngersoll,  2004).  No  matter  how  the  program  is 
structured,  providing  access  to  mentors  to  all  teachers  can  be  problematic.  Providing  an  appropriate  match  of  novice 
and  mentor  can  pose  a challenge.  These  programs  can  also  be  a great  strain  on  a system’s  resources  (Schlager, 

Fusco,  Koch,  Crawford,  & Phillips,  2003). 

Despite  the  challenges,  the  efforts  are  worthwhile.  Smith  and  lngersoll  (2004)  found  that  the  largest 
reductions  in  attrition  were  connected  to  schools  that  provided  activities  that  put  new  teachers  in  collaborative 
networks  with  more  experienced  peers. 

Teacher  Professional  Development 

Teacher  professional  development  differs  from  professional  development  found  in  other  fields.  Schlager, 
Fusco,  & Schank  ( 1998)  credit  this  to  the  circumstances  that  (a)  teachers  are  isolated,  (b)  teacher  professional 
development  takes  place  as  formal,  highly-structured  activities  that  are  not  relevant  to  the  daily  work  of  teachers, 
and  ( c)  often  uses  the  “train-the-trainer”  model  which  often  does  not  meet  the  needs  of  most  teachers.  Most  teacher 
professional  development  activities  are  conducted  as  inservice  days  or  workshops.  This  piecemeal  approach  does  not 
allow  for  a continuous  and  customized  plan  for  effective  teacher  professional  development  (Schlager,  Fusco,  & 
Schank,  2002). 

For  this  study  we  choose  to  view  professional  development  as  a holistic  process.  We  have  chosen  to  adopt 
the  definition  given  by  Schlager  and  Fusco  (2004),  “Professional  development  is  viewed  as  a career-long,  context- 
specific,  continuous  endeavor  that  is  guided  by  standards,  grounded  in  the  teacher’s  own  work,  focused  on  student 
learning,  and  tailored  to  the  teacher’s  stage  of  career  development”  (p.  5). 
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Providing  a holistic  professional  development  program  can  pose  challenges  for  districts.  Teachers  have 
reported  that  they  do  not  value  being  taken  away  from  time  with  their  students  to  attend  workshops  that  are  not 
content-specific  enough  to  be  meaningful  (Alvarado,  2006).  Because  time  is  often  taken  away  from  instruction  to 
conduct  professional  development  activities,  school  systems  should  make  every  effort  to  ensure  those  activities  are 
high  quality. 

Corcoran,  Shields,  and  Zucker  ( 1998),  suggest  that  if  professional  development  programs  are  left 
unchanged,  districts  will  have  to  make  a trade-off:  either  work  with  large  numbers  of  teachers  superficially  or  small 
numbers  intensively.  They  pose  10  dimensions  that  exemplify  high  quality  professional  development: 

• Put  the  emphasis  on  teacher  knowledge  of  subject  matter. 

• Draw  on  the  knowledge  base  about  teaching. 

• Make  a coherent,  sustained  effort. 

• Have  a provision  of  sufficient  time  and  intensity. 

• Make  opportunities  for  teachers  to  be  active  learners. 

• Help  teachers  make  meaningful  intellectual,  social,  and  emotional  engagement  with  ideas, 
materials,  and  colleagues. 

• Engage  teachers  in  planning,  design,  and  delivery  of  professional  development  activities. 

• Recognize  teachers  varying  expertise,  experience,  and  beliefs. 

• Make  activities  accessible  and  inclusive. 

• Provide  incentives  for  participation,  (p.  15-18) 

Darling-Hammond,  Lieberman,  and  McLaughlin  (1995)  agree  that  the  current  professional  development  process  is 
in  need  of  restructuring.  They  also  view  the  process  not  as  a series  of  workshops  and  inservice  days,  but  as  one  of 
building  reflective  communities.  They  suggest  that  traditional  inservice  be  replaced  by  opportunities  for  knowledge 
sharing.  These  opportunities  should  be  afforded  by  blocks  of  time  being  set  aside  specifically  for  this  purpose.  They 
also  suggest  that  these  knowledge  sharing  times  be  encouraged  by  district  leadership  and  that  the  leadership  provide 
the  resources  necessary  to  accomplish  this  form  of  professional  development. 

These  collaborative  forms  of  professional  development  facilitate  teachers  becoming  leaders.  Riel  and 
Becker  (2000)  and  Gutierrez  and  Basile  (2007)  describe  strong  teacher  relationships  as  being  essential  to  teacher 
leadership.  This  transformation  to  leadership  embodies  the  transition  to  holistic,  collaborative  professional 
development  by  allowing  teachers  to  provide  guidance  to  their  peers. 

This  reformed  version  of  professional  development  can  aid  in  teacher  retention  by  giving  teachers  an 
equitable  voice  (Gutierrez  & Basile,  2007).  The  collaborative  process  can  help  reduce  the  reported  isolation  felt  by 
teachers  (Darling-Hammond,  et  al.,  1995).  With  this  in  mind,  several  strategies  should  be  implemented  for 
professional  development  with  a goal  of  teacher  retention.  Studies  (Corcoran,  et  al.,  1998;  Darling-Hammond,  et  al., 
1995;  Gutierrez  & Basile,  2007;  Schlager,  et  al.,  1998)  have  revealed  some  of  the  most  important  characteristics  of 
this  type  of  professional  development.  These  characteristics  are  (1)  professional  development  is  an  ongoing  process, 
(2)  knowledge  sharing  among  teachers  is  necessary,  (3)  time  allowance  for  collaboration  should  be  incorporated, 
and  (4)  teachers  should  be  engaged  in  the  planning  of  activities. 

Teacher  induction  and  professional  development  can  both  aid  school  systems  with  teacher  retention.  Both 
induction  and  professional  development  work  better  when  teaches  work  in  collaborative  groups.  When  teachers 
begin  working  together  toward  common  goals,  they  form  a community. 

Communities  of  Practice 

Communities  of  practice  were  first  introduced  by  Lave  and  Wenger  in  1991.  They  were  expounded  upon 
by  Wenger  in  1998.  Wenger  describes  communities  of  practice  as  being  three-fold.  Lor  communities  to  be 
communities  of  practice,  they  must  have  a “mutual  engagement,  a joint  enterprise,  and  a shared  repertoire”  (p.  73). 
Brown  and  Duguid  ( 1991,  2000)  also  stress  the  importance  of  the  word  practice.  They  posit  that  practice  is  central  to 
the  concept  and  that  the  purpose  of  a community  of  practice  is  the  activity  of  getting  work  done. 

This  “work”  that  needs  getting  done  by  the  community  of  practice  can  be  general  or  specific.  One  type  of 
community  of  practice,  the  bounded  community,  is  formed  in  a structured  setting  and  begins  with  specific  work.  The 
setting  is  typically  a class.  These  communities  are  bounded  by  expectations,  as  well  as  a timeframe.  Bounded 
communities  go  through  three  phases:  initiation,  participation,  and  closure.  It  is  important  to  note  that  part  of  closure 
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may  be  the  formation  of  an  unbounded  community  (Wilson,  Ludwig-Harmon,  Thorman,  & Dunlap,  2004).  For  this 
type  of  community  to  fit  our  key  principles  for  teacher  induction  and  professional  development,  they  must  form  an 
unbounded  community. 

Designing  for  Communities  of  Practice 

Although  teachers  often  work  together  for  a common  purpose,  they  do  not  necessarily  form  communities  of 
practice.  Schlager  and  Fusco  (2004)  state  that  communities  of  practice  should  not  be  created  separate  from  the 
professional  community.  Wenger,  McDermott,  and  Snyder  (2002)  give  seven  design  principles  to  form  exciting, 
relevant,  and  valuable  communities  of  practice:  1)  design  for  evolution;  2)  open  a dialogue  between  inside  and 
outside  perspectives;  3)  invite  different  levels  of  participation;  4)  develop  both  public  and  private  community 
spaces;  5)  focus  on  value;  6)  combine  familiarity  and  excitement;  and.  7)  create  a rhythm  for  the  community. 
Following  these  principles,  a community  of  practice  can  be  successfully  designed  for.  Flowever  the  bigger  challenge 
may  be  sustaining  the  community. 

Participation  and  adaptability  are  key  to  the  sustainability  of  communities  of  practice.  Active  participation 
within  the  community  is  vital  to  its  success  (Wenger,  McDermott,  & Snyder,  2002;  Wilson,  et  al.,  2004). 
Participation  in  a community  of  practice  is  voluntary.  So  to  sustain  the  community,  it  must  be  a place  where  people 
want  to  be.  It  must  be  exciting,  relevant,  and  valuable  (Wenger,  et  al.,  2002).  To  remain  exciting,  relevant,  and 
valuable,  the  community  must  adapt  to  new  members  and  new  circumstances  (Brown  & Duguid,  1991). 

Social  Networks 

Because  communities  of  practice  must  adapt  to  survive,  we  want  to  look  at  social  networks  as  a means  of 
making  new  connections  for  the  community  of  practice.  Penuel  and  Riel  (2007)  define  a social  network  as  “a  set  of 
people  and  the  relationships  among  them”  (p.  61 1).  These  networks  provide  many  advantages  such  as  the  promotion 
of  collaboration,  building  new  relationships,  and  bettering  communication  skills  (Cross,  Borgatti,  & Parker,  2002, 
Jones  & Conceicao,  2008).  They  also  provide  access  to  experts  and  other  help  outside  a member’s  immediate  circle 
(Penuel  & Riel,  2007). 

Social  networks  support  collaboration,  which  is  important  for  job  satisfaction  and  performance  (Cross,  et 
al.,  2002).  They  also  aid  in  the  promotion  of  personal  development  (Jones  & Conceicao,  2008).  This  increased 
satisfaction  may  lead  to  higher  retention  rates  for  new  teachers. 

Limitations 

As  with  induction  and  professional  development,  access  to  experts  and  time  are  obstacles  in  communities 
of  practice.  Penuel  and  Riel  (2007)  state  that  building  in  time  for  experts  to  help  is  crucial,  especially  if  funding  is 
scarce.  To  combat  the  scarcity  of  time  and  access  to  experts,  we  must  look  at  another  option  for  supporting 
communities  of  practice. 

Online  Communities 

Professional  networks  among  teachers  are  a growing  trend  (Schlager,  Farooq,  Fusco,  Schank,  & Dwyer, 
2009).  Traditionally,  teacher  induction  and  professional  development  have  taken  place  in  face-to-face  groups. 
Communities  of  practice  have  also  taken  the  face-to-face  format  to  meet.  However,  with  the  prevalence  and 
accessibility  of  the  Internet,  we  should  explore  how  web-based  tools  can  be  used  to  facilitate  communities  of 
practice  with  the  purpose  of  induction  and  professional  development. 

Wenger,  White,  Smith,  & Rowe  (2005)  point  out  that  technology  does  not  make  a community.  This 
illustrates  the  importance  of  keeping  the  focus  on  the  community  of  practice  and  not  on  the  technology.  Technology 
can  provide  access  to  experts  that  are  not  located  in  a specific  school  or  even  region.  It  can  enable  more  time  to 
access  the  community  outside  of  working  hours.  It  can  provide  many  tools  to  facilitate  interactions  within  the 
community.  However,  as  stated  in  previous  sections,  participation  and  adaptability  are  key  to  sustaining 
communities.  The  tools  must  allow  for  and  encourage  participation  and  adaptability  or  the  community  risks  certain 
failure. 

Benefits  of  Online  Communities  of  Practice 

Online  communities  provide  members  with  access  to  community  resources  24  hours  a day,  seven  days  a 
week  (Grove,  Odell,  & Strudler,  2006).  By  sharing  or  posting  resources  in  an  online  community,  the  time  constraints 
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of  meeting  during  work  hours  are  lessened  if  not  completely  lifted.  In  Fletcher,  Tobias,  and  Wisher’s  (2007)  work 
with  advanced  distributed  learning,  they  found  it  was  beneficial  to  have  materials  readily  accessible  for  teachable 
moments.  The  same  concept  applies  to  storing  resources  in  the  online  community. 

Another  benefit  of  online  tools  is  the  ties  that  are  formed.  It  is  not  always  the  intended  audience  that 
benefits  from  the  sharing  of  resources  in  the  online  community  (Schlager,  et  al.,  2009).  Others  may  benefit  at  a 
different  time  or  in  a different  situation  from  materials  that  are  shared. 

Online  communities  of  practice  also  allow  for  access  to  experts  that  may  not  be  located  in  physical 
proximity  to  a school.  This  is  especially  beneficial  when  funding  is  not  available  to  bring  experts  to  the  location. 
Many  teachers  find  themselves  in  situations  where  they  are  the  only  person  in  their  area  of  expertise.  Online 
communities  allow  them  to  share  resources  with  experts  or  peers  who  share  their  situation. 

Limitations  of  Online  Communities  of  Practice 

Freeing  up  time  and  access  to  experts  are  great  benefits  of  online  communities  of  practice.  But  despite 
these  benefits,  online  communities  are  still  not  well  understood  (Schlager,  et  al.,  2009).  Even  though  online 
communities  of  practice  allow  more  options  for  time  to  participate,  a survey  by  Fusco,  Gehlbach,  & Schlager  (2000) 
found  that  lack  of  time  was  still  a barrier  to  participation  for  some  members  of  an  online  community  of  practice. 

Time  is  not  the  only  resource  that  limits  the  adoption  of  online  communities.  Lack  of  funding  and  human 
capital  also  pose  barriers  (Farooq,  Schank,  Harris,  Fusco,  & Schlager,  2007;  Fusco,  et  al.,  2000).  Online  platforms 
often  have  a subscription  fee  and  training  users  on  the  platform  can  also  incur  costs.  Also,  a facilitator  is  often 
necessary  to  preside  over  groups,  at  least  initially  (Spitzer  & Wedding,  1995). 

Another  limitation  is  the  lack  of  tools  to  assess  online  communities  of  practice.  Current  measures  and 
technologies  do  not  adequately  measure  or  characterize  the  communication  in  these  groups  (Schlager,  et  al.,  2009; 
Tatar,  Gray,  & Fusco,  2002).  Better  assessment  tools  need  to  be  created  for  studying  online  communities  of  practice. 

The  user-friendliness  of  the  online  community’s  platform  can  also  be  an  obstacle.  If  the  site  does  not  meet 
the  end  user’s  needs  (Farooq,  et  al.,  2007)  or  is  difficult  to  navigate  (Fusco,  et  al.,  2000;  Schank,  et  al.,  2002)  users 
will  become  discouraged  from  investing  time  and  energy  into  the  online  community  of  practice. 

Perhaps  the  largest  obstacle  in  implementing  online  communities  of  practice  is  developing  a critical  mass 
of  participants  (Barab,  McKinster,  & Moore,  2001;  Farooq,  et  al.,  2007;  Hebert,  Clift,  & Wennerdahl,  2008).  Active 
participation  is  key  to  sustaining  any  community  of  practice.  This  becomes  even  more  apparent  in  an  online 
community  (Spitzer  & Wedding,  1995).  When  members  log  in  to  their  online  community  and  no  one  is  ever  there  to 
collaborate  with,  they  lose  interest  and  the  community  as  a whole  loses  value. 

Designing  for  Online  Communities  of  Practice 

In  a study  by  Schlager,  Fusco,  Koch,  Crawford,  and  Phillips  (2003),  they  asked  how  a technology- 
supported  teacher  network  could  be  implemented  and  scaled  up  to  meet  teachers'  professional  support  needs.  We 
also  seek  to  answer  that  question  in  this  study.  While  there  is  not  a one  size  fits  all  answer  to  the  question,  the 
literature  provides  us  with  direction. 

To  mitigate  the  limitations  of  online  communities  of  practice,  a great  deal  of  thought  must  go  in  to  the 
design  of  the  online  platform  (Spitzer  & Wedding,  1995).  Koch  and  Fusco  (2008)  give  three  phases  online 
communities  of  practice  go  through:  getting  started,  modeling  and  scaffolding,  and  maturing.  It  is  important  that  the 
platform  meet  the  needs  of  the  community  at  all  three  levels. 

Bringelson  and  Carey  (2000)  encourage  designers  of  platforms  to  be  aware  of  the  community  for  which 
they  are  designing.  The  designer  must  know  who  the  community  members  are  and  why  they  are  engaged  in  the 
community  (Fusco,  Gehlbach,  & Schlager,  2000).  Bringelson  and  Carey  (2000)  give  us  five  guiding  points  to  use 
when  assessing  the  community.  When  designing  a virtual  venue,  one  should  know  the  community’s: 

• previous  exposure  to  technology, 

• access  to  new  and  developing  technologies, 

• time  and  rhythm  of  accessing  the  venue, 

• breadth  and  depth, 

• and  sustainability  of  the  interactions,  (p.  63) 

The  technology  also  needs  to  support  a climate  of  comfort  and  trust  for  the  members  of  the  community. 
This  includes  generating  a feeling  of  togetherness  that  lasts  over  time  and  across  distance  (Wenger,  White,  Smith  & 
Rowe,  2005).  Allowing  for  conditions  that  support  critical  thinking  and  constructive  criticism  promote  more  risk- 
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taking  and  response.  This  in  turn  allows  for  groups  to  be  innovative  (Tatar,  Gray,  & Fusco,  2002).  This  risk-taking, 
responsiveness,  and  innovation  build  the  excitement  and  add  value  to  the  online  community  of  practice.  These 
characteristics  add  to  the  sustainability  of  the  group. 

Another  key  component  for  designing  sustainable  platforms  is  providing  access  to  support.  Farooq,  Schank, 
Flarris,  Fusco,  and  Schlager  (2007)  revealed  four  design  interventions  that  contributed  to  the  sustainability  of  their 
online  platform.  The  four  interventions  are  providing  users  with  contact  and  bug  reporting  forms,  a forum  for  group 
members  to  discuss  needed  features  for  the  platform,  a published  task  list  so  users  could  see  what  developments 
were  in  the  works  and  a timeframe  for  their  implementation,  and  a help  desk. 

Much  more  research  needs  to  be  done  on  the  effectiveness  of  online  platform  designs.  First,  tools  will  have 
to  be  developed  to  accurately  project  the  value  communities  receive  from  their  online  platforms.  Fusco,  et  al.  (2000) 
also  suggest  the  need  to  know  if  there  is  a correlation  between  log  ins  on  a platform  and  perceived  gains  of  the  users. 
We  expect  many  more  research  questions  in  the  near  future  as  the  use  of  online  communities  of  practice  continues  to 
expand. 

Summary 

Online  communities  of  practice  can  help  slow  teacher  attrition  by  lessening  the  isolation  felt  by  early  career 
educators  (Spitzer  & Wedding,  1995).  When  used  for  teacher  induction  and  professional  development,  online 
communities  of  practice  provide  educators  with  access  to  experts,  peers,  and  high-quality  resources  without  the 
inhibitors  of  time  and  physical  limitations.  While  there  is  much  research  that  needs  to  be  done  to  validate  the  use  of 
online  platforms  for  teacher  retention,  the  literature  shows  that  their  use  is  a viable  option  for  many  school  districts. 

Case  Study 

The  University  of  Memphis  is  one  of  three  TLINC  project  sites.  The  focus  of  the  Memphis  site  is  to  form 
online  communities  of  practice  for  student  teachers.  During  the  2008-2009  school  year,  preservice  teachers  at  the 
university  were  assigned  to  a mentoring  group  during  their  semester  of  student  teaching.  The  student  teachers 
connected  with  their  groups  online  using  the  website  Tapped  In  (http://tappedin.org).  Each  group  consisted  of  a 
mentor  and  an  average  of  six  student  teachers  that  share  a common  focus  area.  The  mentors  were  either  university 
faculty  or  a classroom  teacher  who  has  been  teaching  for  more  than  three  years  and  has  been  recommended  for  the 
program  by  their  supervisor  or  university  faculty.  170  student  teachers  and  20  mentors  participated  in  the  project. 

During  the  year,  the  groups  met  for  a synchronous  chat  at  least  three  times  dining  the  semester.  The 
average  chat  lasted  about  45  minutes.  Mentors  facilitated  the  conversation  dining  each  session.  The  TLINC  research 
team  did  not  assign  discussion  topics.  The  mentors  chose  topics  they  perceived  to  be  beneficial,  and  topics  emerged 
from  concerns  voiced  by  the  student  teachers.  The  mentors  submitted  transcripts  of  each  chat  to  the  research  team. 

Methodology 

The  approach  to  inquiry  chosen  for  this  study  was  a single,  within-site,  case  study.  The  issue  addressed  was 
the  isolation  felt  by  new  teachers.  The  bounded  case  consisted  of  students  completing  their  student  teaching 
semester  during  the  2008-2009  school  year  at  the  University  of  Memphis.  Data  for  this  study  was  in  the  form  of 
collected  transcripts  of  the  small  group  discussions.  105  transcripts  were  collected  from  August  2008  until  May 
2009.  Maximum  variation  was  sought  by  including  all  of  the  submitted  transcripts  in  the  analysis  to  identify 
important  common  patterns  (Cresswell,  2007).  Two  researchers  independently  reviewed  each  transcript  and  coded 
the  comments  into  categories.  The  researchers  then  compared  and  discussed  the  categories.  Common  themes 
emerged  during  this  process. 

During  this  study,  data  was  also  collected  using  electronic  surveys  and  focus  group  interviews.  However, 
the  focus  of  this  article  is  the  conversations  of  student  teachers  in  online  communities.  Therefore,  only  the  data  from 
the  conversation  analysis  is  included. 


Findings 

The  topics  discussed  in  the  online  communities  varied  between  groups.  All  groups  included  general 
greetings  and  small  talk  at  the  start  of  their  sessions.  This  usually  occurred  when  two  or  three  participants  had 
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logged  on  and  were  waiting  for  the  other  participants  to  join  them.  In  early  chats,  some  technical  issues  were 
discussed.  Some  participants  shared  issues  they  had  logging  into  the  system  or  finding  their  assigned  group  room. 
Comments  concerning  technical  issues  decreased  dramatically  as  the  semester  progressed. 

Although  the  group  members  were  pursuing  various  teaching  specialties,  (i.e.  special  education,  elementary 
education,  secondary  education,  etc.)  they  shared  many  common  topics  of  discussion.  Three  key  themes  emerged 
from  this  analysis.  The  emergent  themes  were  The  Work  of  Teaching,  The  Teacher  as  a Professional,  and  Readiness 
to  Teach.  These  themes  are  categorized  in  the  following  table. 


Themes 

Sub-Categories 

The  Work  Of  Teaching 

• 

Lesson  Planning 

• 

Teaching  Strategies 

• 

Classroom  Management 

• 

Time  Management 

The  Teacher  as  a Professional 

• 

Supervisory  Issues 

• 

Culture  of  the  School 

• 

Sharing  Resources 

• 

About  the  Field 

Readiness  to  Teach 

• 

Preparedness  (as  related  to  coursework) 

• 

Emotions 

• 

Relating  to  Students 

• 

Experiences  in  the  School 

• 

Securing  Employment 

Implications 

Based  on  these  emergent  themes,  the  structure  of  the  University  of  Memphis'  TLINC  project  will  change. 
Beginning  in  January  2010,  mentors  will  base  each  of  their  three  required  synchronous  chats  on  one  of  the  three 
emergent  themes.  This  will  give  the  groups  more  focus  and  allow  mentors  to  better  prepare  for  the  chats.  Students 
will  also  have  a better  idea  of  what  contributions  they  can  bring  to  the  group  each  month.  Asynchronous  chats  using 
a discussion  board  will  be  used  to  address  concerns  that  the  students  have  that  do  not  fit  one  of  the  themes.  The 
discussion  boards  will  also  be  used  when  students  have  a concern  that  needs  to  be  addressed  before  the  next 
scheduled  chat  time. 

The  findings  of  this  study  will  also  be  shared  with  the  University’s  Office  of  School  Based  Clinical 
Practice.  This  office  conducts  workshops  and  sessions  for  student  teachers.  Knowing  these  themes  may  benefit  them 
as  they  plan  future  workshops  for  student  teachers. 

Future  studies  are  also  being  planned  based  on  the  findings  of  this  analysis.  Member  checking  of  the 
emergent  themes  is  currently  being  conducted.  The  data  collected  from  the  transcripts  must  now  be  triangulated  with 
the  results  of  the  electronic  surveys  and  focus  group  discussions.  Hopefully  the  complete  picture  provided  by  this 
triangulation  of  data  will  tell  us  if  this  process  met  our  goals  of  reducing  the  isolation  felt  by  new  teachers  and 
improving  their  proficiency  as  they  enter  the  classroom.  A follow  up  study  needs  to  be  done  with  the  students  who 
participated  in  this  project  as  they  enter  their  first  years  of  teaching.  It  will  be  important  to  know  if  they  continue  to 
use  online  communities  of  practice  for  support.  It  will  also  be  significant  to  know  what  their  perceived  benefits  of 
these  groups  are,  especially  if  they  lessen  their  feelings  of  isolation. 
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Introduction 

Distance  education  evokes  strong  reactions  for  many  in  the  field  of  higher  education.  When  considering 
distance  education  options,  students  search  like  consumers.  Students  search  for  courses  convenient  to  their  schedule, 
budget,  and  lifestyle.  With  more  non-traditional  students  enrolling  in  higher  education,  distance  education  is  being 
implemented  to  help  meet  student  demand,  and  higher  education  is  becoming  a proving  ground  for  new  forms  of 
content  delivery. 

For  administrators,  distance  education  may  be  seen  as  a source  of  increased  revenue.  This  can  lead  to  both 
positive  and  negative  repercussions.  In  a positive  sense,  education  is  available  to  more  students.  In  a negative  sense, 
distance  education  can  be  viewed  as  a means  of  generating  more  revenue  with  less  regard  for  quality. 

These  relatively  new  types  of  demands  involve  the  rapid  movement  towards  distance  education. 

Distance  Education  Defined 

The  definition  of  distance  education  varies  across  and  even  within  universities.  With  mislabeled  course  descriptions, 
students  are  unable  to  determine  their  ability  to  succeed  in  the  course.  Only  certain  personalities  have  been  proven 
successful  in  distance  education  courses  (Heffner  & Cohen,  2005).  Successful  students  are  self-motivated  and  able 
to  work  independently.  At  the  same  time,  these  students  are  confident  enough  to  receive  minimal  feedback  from  the 
instructor  at  times.  Perhaps  most  important,  the  successful  student  is  competent  with  Internet,  e-mail, 
communication  on-line,  and  minimal  typing  and  writing  skills. 

Unfortunately,  the  decisions  for  course  descriptions  are  often  determined  by  each  college  within  the 
university  or  by  a particular  administrator  instead  of  the  professor  that  is  responsible  for  the  course.  Therefore, 
without  a set  definition,  confusion  takes  place  for  all  those  involved.  For  example,  a web  assisted  class  in  the  college 
of  business  may  be  a face-to-face  course  with  a Learning  Management  System  (LMS)  for  videos  and  PowerPoints 
available.  However,  a web  assisted  course  at  the  same  university  in  the  college  of  history  may  meet  face-to-face 
only  five  times  the  semester  and  have  a LMS  for  discussion  boards,  assignments,  etc.  Students  may  not  be  aware  of 
the  type  of  course  in  which  they  have  enrolled  until  the  first  day  of  class. 

The  most  common  terms  that  apply  when  discussing  distance  education  include  Compressed  Video  Network  (CVN), 
Web  Assisted,  or  Hybrid.  To  help  minimize  this  confusion  for  the  purpose  of  this  discussion,  CVN,  web  assisted, 
hybrid,  and  distance  education  will  each  be  defined. 

Compressed  Video  Network  operates  with  “narrow  bandwidth  communications”  (Schlosser  & Simpson, 
2005).  CVN  is  used  to  teach  students  at  more  than  one  location  simultaneously.  To  be  successful  using  this  method, 
the  instructor  is  required  to  be  creative.  All  students  are  not  in  the  same  room  with  the  instructor,  in  some  cases,  the 
student  may  be  the  only  one  in  the  room.  However,  the  students  can  interact  with  each  other  and  the  teacher  through 
visual  and  verbal  communications.  Although  the  instructor  cannot  always  be  in  the  classroom  with  the  students 
he/she  can  view  the  body  language  and  facial  reactions  of  the  students,  depending  on  the  equipment  available.  If 
proper  equipment  such  as  large  screens  is  not  available,  instructors  will  not  be  able  to  view  or  relate  to  the  class  at 
alternative  sites.  Each  individual  class  is  connected  and  treated  “as  part  of  the  extended  class”  (Hanna,  1998).  The 
students  and  instructors  meet  at  regularly  scheduled  times  but  at  the  convenience  of  various  locations. 

CVN  seems  to  be  a perfect  blend  between  the  traditional  classroom  and  distance  education.  However,  some 
universities  found  that  “using  compressed  video  as  the  single  medium  of  delivering  distance  education”  (Weber, 
1996)  was  not  as  efficient  as  hoped.  Instructors  face  issues  when  diving  into  the  classroom.  A few  examples 
observed  include  the  equipment  turning  off  in  the  middle  of  a student  question  or  shutting  down  if  the  students  are 
not  moving  in  the  off-site  classroom.  Another  issue  includes  the  instructor’s  voice.  A soft  spoken  or  strongly 
accented  instructor  would  be  difficult  to  understand.  In  the  case  observed,  the  microphone  was  placed  on  a podium 
restricting  movement  and  requiring  the  soft  spoken  instructor  to  speak  directly  into  it.  Hands-on  information  do  not 
always  translate  well  through  this  medium.  Furthermore,  some  instructors  complain  because  this  type  of  instruction 
limits  their  freedom  of  movement 


478 


Some  issues  for  administrators  to  consider  when  beginning  a CVN  program  include  scheduling  between 
campuses,  grading  systems,  availability  of  books  and  materials,  student  advising,  and  faculty  training  (Weber, 

1996).  When  scheduling  classes,  the  course  must  accommodate  students  on  all  campuses  involved.  The  students 
should  be  advised  on  what  is  expected  in  the  course  and  where  and  when  the  books  will  be  available.  Lastly,  the 
faculty  must  be  trained  on  the  equipment.  In  the  event  of  a technical  problem,  the  instructor  should  be  able  to  handle 
minor  operating  problems  without  calling  for  technology  assistance.  When  communicating  with  students,  the 
instructor  will  need  to  rely  on  e-mail  for  the  off-campus  site(s).  It  will  be  the  best  way  to  keep  in  constant  contact 
and  monitor  progress.  As  far  as  the  instruction  method,  the  lecture  method  does  not  work  well  in  this  environment. 
However,  “multimedia  presentations”  have  been  proven  effective  (Weber,  1996).Nonetheless,  some  instructors  are 
unable  to  interact  with  students  without  visual.  Tasks  that  require  hands-on  manipulations  do  not  work  well  through 
CVN.  Instructors  also  discovered  teaching  at  each  site  throughout  the  course  helped  to  develop  relationships  with 
the  students. 

Another  type  of  evolving  non-traditional  classroom  is  the  web-assisted  or  hybrid  course.  The  web-assisted 
or  hybrid  course  is  similar  to  the  traditional  classroom.  Instruction  occurs  face-to-face  with  varying  degrees  of 
technology  implemented.  Often,  the  students  can  access  the  information  through  a wiki,  blog,  or  learning 
management  system  (LMS)  such  as  Blackboard.  According  to  the  2006-2007  National  Center  for  Education 
Statistics  (NCES)  report,  35%  of  2 and  4-year  institutions  reported  offering  hybrid  courses  (Parsad  & Lewis,  2008). 
At  the  same  time,  the  number  of  students  enrolled  in  hybrid  classes  was  just  over  one  million  (Parsad  & Lewis, 
2008).  Several  examples  for  “extended  traditional  universities”  are  available  (Hanna,  1998).  One  includes 
Washington  State  University  (WSU)  and  their  Web  University.  WSU  foresees  the  Web  University  “courses  suited 
for  electronic  export  via  satellite,  Internet,  or  a K-20  network”  (Hanna,  1998).  (Note  the  word  suited.)  Similarly,  the 
University  of  Wisconsin  has  developed  the  Center  for  Learning  Innovation,  now  known  as  UW  Learning 
Innovations,  in  order  to  “develop  and  commercialize”  the  university  programs  and  software  (Hanna,  1998).  WSU 
and  UW  predicted  the  surge  of  CVN,  hybrid,  on-line  and  cooperative  programs  with  K-12  schools  more  than  ten 
years  ago.  Looking  at  this  prediction  now,  where  will  these  programs  be  in  ten  years  more? 

To  compare  a hybrid  course  to  traditional  face-to-face  course,  a study  was  conducted  with  80  psychology 
students  (DeNeui  & Dodge,  2006).  It  focused  on  the  integration  of  Blackboard  in  a face-to-face  course.  The  students 
were  not  required  to  use  Blackboard  to  pass  the  course;  however,  all  course  documents,  assignments, 
announcements,  and 

detail  the  number  of  times  each  student  visited  the  site  (DeNeui  & Dodge,  2006).  They  found  the  “females  used 
Blackboard  significantly  more  than  did  males"  (DeNeui  & Dodge,  2006).  In  relation  to  this  result,  the  females  had  a 
significantly  higher  final  grade  than  the  males.  Overall,  the  researchers  only  found  a “small  but  significant 
correlation”  between  Blackboard  usage  and  final  grade  (DeNeui  & Dodge,  2006).  Another  study  on  the  correlation 
of  grades  and  students’  behavior  was  performed.  Its  results  were  significant  for  the  154  undergraduate  psychology 
students  involved  (Heffner  & Cohen,  2005).  The  students  reported  a 90%  daily  internet  usage  with  27%  of  the 
students  stating  the  course  caused  an  increase.  The  students  were  using  the  Internet  to  access  the  courses  the 
information.  According  to  the  researchers,  the  students  who  passed  the  coursed  “more  frequently  accessed  the 
Answers  to  Reading  and  Lecture  Questions”  portion  of  the  website  (Heffner  & Cohen,  2005).  Also,  there  was  a 
significant  correlation  between  the  final  exam  score  and  the  Answers  to  Reading  and  Lectures  Questions  which  was 
viewed  as  a study  guide  (Heffner  & Cohen,  2005).  An  interesting  pattern  from  the  data  was  the  females  accessed  the 
information  more  frequently  than  the  males.  Nearly  everyone  reported  using  the  Internet  daily  to  gain  information. 

Distance  education  is  “a  field  of  education  that  focuses  on  the  pedagogy  and  andragogy,  technology,  and 
instructional  systems  design  that  aim  to  deliver  education  to  students  who  are  not  physically  "on  site"  (Distance 
Education,  2009).  In  comparison,  Schlosser  and  Simonson  define  it  as  “institution-based,  formal  education  where 
the  learning  group  is  separated,  and  where  interactive  telecommunications  systems  are  used  to  connect  learners, 
resources,  and  instructors”  (Schlosser  & Simpson,  2005).  Distance  education  occurs  when  students  are  connected 
most  often  by  the  internet  through  a learning  management  system.  During  the  2006-2007  school  term,  9.39  million 
students  enrolled  in  distance  education  courses  (Parsad  & Lewis,  2008). The  instructor  delivers  assignments, 
provides  discussions,  projects,  and  administers  tests. 

Washington  State  University  (WSU)  developed  Distance  Degree  Programs  (DDP)  in  order  to  meet  the 
needs  of  distance  education  (Kendall,  2005).  In  1992,  WSU  began  the  DDP  with  fifty-seven  students.  Now  the  DDP 
has  graduated  1,400  with  bachelor’s  degree  and  enrolls  an  estimated  5,500  students  each  semester  (Kendall,  2005). 
The  DDP  at  WSU  ensures  each  course  meets  internal  integrity  or  a face-to-face  course  will  contain  the  same  content 
as  an  online  course.  The  DDP  acts  as  a portal  for  students  to  enter  their  courses,  gain  information  about  advising, 
grades,  etc.  At  the  end  of  each  semester,  the  students  are  given  an  evaluation  for  feedback.  Their  advisors  encourage 
them  of  save  a copy  for  records.  Other  services  offered  from  DDP  include  tutoring,  live  chatting  with  forums, 
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communication  areas,  and  book  rental  sites  (Kendall,  2005).  Each  of  these  services  have  been  tried  and  proven  at 
WSU.  It  is  similar  to  the  University  of  Memphis  student  MyMemphis  portal.  My  Memphis  portal  includes  has  links 
for  registration,  accounts,  admissions,  advising,  umdrive,  etc.  Individual  institutions  should  determine  their  and  their 
students’  needs. 


Growth  and  Success  in  Distance  Education 

The  United  States  Department  of  Education  has  monitored  the  growth  of  distance  education.  According  to  the 
National  Center  for  Education  Statistics,  the  number  of  distance  education  courses  is  growing  rapidly.  During  the 
2006-07  school  year,  66%  of  the  4,200  postsecondary  institutions  offered  a form  of  distance  education  course. 
Interestingly,  the  most  widely  accepting  of  the  distance  education  was  the  public  two  year  colleges.  Almost  all  of 
these  community  based  colleges  allowed  for  online  courses  and  degree  programs.  Surprisingly,  the  schools  lagging 
behind  were  the  private  universities  and  colleges. 

By  spring  2009,  Arkansas  and  Tennessee  have  already  observed  growth  of  distance  education  courses 
offered  on  both  college  and  university  campuses.  According  to  web-based  research  gathered  from  institutions  in 
Arkansas  and  Tennessee,  the  national  numbers  have  increased  by  comparing  the  current  state  numbers.  In  Arkansas, 
there  are  44  public  or  private  universities  and  community  colleges.  At  this  time,  forty-one  institutions  offer  online 
courses  (Arkansas  Department  of  Higher  Education,  2009).  These  online  courses  are  those  which  are  taught  using  a 
LMS.  In  comparison,  Tennessee  has  46  public  or  private  universities,  community  colleges,  and  Tennessee  Technical 
Colleges  (Tennessee  Higher  Education  Commission,  2009).  Each  institution  offers  online  courses. 

As  the  numbers  indicate,  distance  education  is  becoming  the  educational  method  of  choice.  Distance 
education  has  become  popular  “for  working  adults,  for  persons  in  locations  without  immediate  access  to  formal 
education  facilities,  and  for  persons  with  disabilities”  (Milligan  & Buckenmeyer,  2008)  For  example,  working  adults 
may  include  first  time  degree  seekers  or  degree  seekers  who  are  earning  their  graduate  degrees.  While  these  students 
are  more  likely  to  appreciate  the  convenience  of  distance  education,  all  of  the  students  may  not  have  the  necessary 
skills  required  for  the  course(s).  These  categories  of  students  have  become  known  as  the  non-traditional  students.  To 
meet  their  specific  needs,  administrators  have  begun  developing  degree  and  certificate  programs  online  exclusively. 
An  estimated  1 1,200  degree  and  certificate  programs  have  been  designed  for  this  specific  purpose  by  2006-2007 
(Parsad  & Lewis,  2008).  Of  those  online  opportunities,  66%  were  repotted  as  a degree  program  (Parsad  & Lewis, 
2008).  At  Arkansas  State  University,  graduate  students  can  earn  a MSE  in  Education  Theory  and  Practice  (a  non- 
licensure program)  through  online  courses. 


Future  Issues 

Issues  such  as  convenience,  increased  class  sizes,  increased  revenues,  and  economical  concerns  are  among 
the  issues  causing  these  trends  towards  increased  distance  education  classes  and  degree  programs.  When  trying  to 
meet  the  needs  of  the  students  the  administrators  look  at  several  items.  According  to  the  NCES  the  most  reported 
items  include,  flexible  schedule  for  students  (68%),  access  for  all  students  (67%),  and  course  availability  (46%) 
(Parsad  & Lewis,  2008).  Administrators  want  to  increase  the  numbers  of  minority  students  while  offering  cost- 
effective  instruction  to  all  students  (Clark,  2009).  According  to  a previously  mentioned  study,  students  who  live  far 
from  campus  are  at  a disadvantage  for  access  to  education.  Therefore,  these  students  might  choose  any  of  the  current 
distance  education  options  available  (Dellana,  Collins,  & West,  2000).  For  example,  students  might  choose  to  attend 
a hybrid  course  with  limited  in  class  instruction  or  a complete  online  course. 

Particular  trends  and  issues  will  be  discussed  as  well  as  appropriateness  of  some  of  the  trends.  Can  any 
class  be  an  appropriate  online  course?  Currently,  administrators  want  to  make  all  types  of  classes  available  on-line. 
This  includes  classes  that  are  theoretical,  field-based,  kinesthetic,  foundations,  and  methodology  courses.  Is  this 
particular  trend  really  where  we  should  go  or  is  it  simply  a means  to  make  money?  The  student-instructor  interaction 
is  a concern  for  many  in  regards  to  distance  education.  Without  the  physical  classroom,  the  interaction  and 
relationship  is  altered  inevitably  in  the  minds  of  some  instructors  (Dellana,  Collins,  & West,  2000).  There  are  mixed 
results  from  research  on  the  overall  effectiveness  of  an  online  methodology  course  (Dellana,  Collins,  & West,  2000). 
Most  show  favorable  results  for  graduate  methodology  courses  while  giving  unfavorable  results  for  undergraduate 
methodology  courses.  This  could  be  related  to  the  type  of  student.  To  help  view  this  more  closely,  a research  study 
was  done  with  199  students.  They  were  enrolled  in  an  identical  course  taught  by  the  same  instructor  either  face-to- 
face  or  face-to-face  with  an  online  component.  Sixty-five  students  were  enrolled  in  the  face-to-face  course  and  135 
were  enrolled  in  the  online  course  (Dellana,  Collins,  & West,  2000).  The  study  found  no  significant  difference 
between  the  two  final  scores  for  the  courses.  However,  in  both  courses,  there  was  a significant  relationship  between 
the  type  of  course,  GPA,  and  absence  rate  (Dellana,  Collins,  & West,  2000) 
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One  type  of  course  discussed  earlier  was  the  hybrid  course.  The  trend  being  researched  is  a hybrid  degree 
program.  Because  the  courses  would  be  traditional  with  online  influence,  the  students  would  have  more  schedule 
flexibility  (Toth,  Fougler,  & Amrein-Beardsley,  2008).  Research  of  nine  hybrid  courses  from  450  students  was 
performed.  To  begin,  the  students  and  instructors  were  given  surveys.  Although  some  students  enjoy  the  hybrid 
method,  expectations  and  course  design  must  be  clearly  communicated  to  students  from  the  beginning.  When 
developing  the  degree  program,  eliminate  any  inconsistencies  from  the  courses  (Toth,  Fougler,  & Amrein- 
Beardsley,  2008). 

To  assist  in  the  determination  of  the  overall  effectiveness  of  the  courses,  each  semester  students  are  asked 
to  evaluate  instructors  and  the  course.  These  evaluations  are  supposed  to  determine  the  instructor’s  ability  to  impart 
knowledge.  However,  there  has  been  “clear  evidence”  these  are  a poor  method  of  evaluation  (Clark,  2009).  With  the 
movement  towards  online  courses,  the  need  to  verify  teaching  ability,  which  is  necessary  for  promotion,  is  becoming 
more  complicated.  It  is  not  as  simple  as  with  a traditional  course.  The  instructor  can  no  longer  invite  the  department 
or  committee  chair  into  the  classroom  to  observe  his  or  her  teaching  ability.  The  instructor  must  produce  evidence  of 
teaching  standards  (Oslington,  2005).  Although  projects  have  been  attempted,  they  have  not  been  proven  successful. 
Another  issue  is  the  ability  to  “attract  good  students”  (Molenda,  2009).  No  matter  the  course  format  this  has  always 
been  an  issue  for  each  department.  However,  technology  departments  understand  the  expense  necessary  for 
equipment  and  the  need  for  students.  The  purchase  of  new  technology  is  expensive.  Therefore,  technology 
departments  must  be  competitive  for  top  students  in  order  to  keep  from  being  given  budget  cuts  (Molenda,  2009). 

Conclusion 

When  designing  the  new  online  courses,  the  curriculum  is  sometimes  allowed  to  suffer.  For  example,  course 
requirements  and  learning  expectations  are  sacrificed  to  meet  “consumer”  needs  (Dellana,  Collins,  & West,  2000). 

In  order  to  be  considered  a quality  course,  it  must  include  pieces  from  the  face-to-face  course,  “effective 
pedagogical  technology,”  and  learning  theory  (Toth,  Fougler,  & Amrein-Beardsley,  2008). 

Teaching  method  includes  collaborative  problem  solving  is  recommended  for  most  higher  education  courses  (Clark, 
2009).  At  the  same  time,  beginning  students  lack  the  skills  required  for  success  in  discovery  learning  and 
collaborative  teams  (Clark,  2009).  Strong  cases  have  been  made  for  guided  discovery  methods  with  demonstrations 
and  immediate  feedback.  Recently  developed  Cognitive  Load  Theory  offers  a solution  to  the  multimedia  instruction. 
There  needs  to  be  more  research  in  this  area.  For  example,  there  has  been  research  on  the  same  instructor  teaching  in 
various  delivery  methods,  but  what  about  the  teacher  that  is  so  dynamic  in  person  that  online  can  not  capture  the 
same  essence?  Again,  the  research  has  primarily  focused  on  classes  in  general  as  if  all  classes  are  the  same.  Do  some 
classes  shine  in  an  on-line  environment  more  than  face-to  face  and  will  which  ones  will  fail  miserable  because  of  the 
particular  content  or  necessary  means  of  delivery.  What  about  when  on-line  is  the  only  option  given  to  students? 
How  does  that  affect  student’s  attitudes  and  successfulness? 
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Introduction 

One  of  the  affordances  of  Internet-based  learning  environments,  from  e-learning  to  online  electronic  games,  is  the 
potential  to  provide  formative  assessment.  Learning  and  instructional  theories  maintain  that  students  come  into  the 
learning  environment  with  preconceptions  of  the  world;  it  is  important  to  conduct  formative  assessment  to  reveal 
their  thinking  and  to  design  activities  to  improve  their  understanding  (Brandsford,  Pellegrino,  & Donovan,  1999). 
Research  indicates  that  formative  assessment  can  indeed  lead  to  strong  learning  gains  (Black  & Wiliam,  1998). 
However,  formative  assessment  is  practiced  infrequently  in  the  classroom  because  it  is  extremely  time  consuming 
and  requires  expertise  that  teachers  usually  do  not  have  (Hunt  & Pellegrino,  2002;  Pedersen,  Arslanyilmaz,  & 
Williams,  2007).  Technology  tools  can  be  designed  to  facilitate  formative  assessment.  There  is  a body  of  literature 
on  formative  assessment  and  computer  tools  designed  to  facilitate  formative  assessment,  but  there  is  little  guidance 
on  how  to  use  technology  to  facilitate  formative  assessment  in  Internet-based  learning  environments. 

This  paper  describes  the  design  of  a web-based  management  system  created  to  facilitate  formative  assessment  in 
Virtual  Environments  for  Learning  (VEL),  which  consists  of  a set  of  computer-based  modules  developed  to  engage 
children  in  student-directed  inquiry.  This  paper  may  be  of  interest  to  researchers  and  practitioners  interested  in 
assessment  in  virtual  learning  environments. 


Formative  Assessment 

There  is  no  consensus  on  the  definitions  of  formative  assessment,  and  different  terms  such  as  “feedback,” 

“formative  evaluation,”  “classroom  evaluation”  have  been  used  interchangeable  with  formative  assessment  (Wiliam 
& Black,  1996).  Formative  assessment  may  be  conducted  using  different  strategies  (McGuire,  2005;  Wiliam,  Lee, 
Harrison,  & Black,  2004)  including  dialog  between  teachers  and  students  about  the  learning  process,  self 
assessment,  peer  assessment,  rich  questioning,  comment-only  marking,  and  criteria  sharing  with  students.  In  spite  of 
the  different  definitions  and  different  strategies  used  to  implement  formative  assessment,  researchers  (Harlen  & 
James,  1997;  Miller,  Bradford,  & Cox,  1998;  Wiliam  & Black,  1996)  seem  to  generally  agree  that  formative 
assessment  differs  from  summative  assessment  in  that  information  about  student  performance  should  be  fed  back  to 
the  student  to  influence  the  learning  process. 

In  schools,  assessment  is  usually  associated  with  accountability  and  standardized  testing.  This  type  of  high-stake 
summative  assessments  is  greatly  valued  by  administration,  yet  researchers  argue  that  these  tests  usually  have  little 
positive  impact  on  student  learning  (Wiliam  et  al.,  2004).  In  contrast,  formative  assessment  can  lead  to  strong 
learning  gains  (Black  & Wiliam,  1998).  For  example,  timely  feedback  reduces  the  learning  time  needed  and  it  better 
motivates  students  (Corbett  and  Anderson  cited  in  Koedinger,  1998).  Two  studies  that  evaluated  the  effectiveness  of 
a web-based  formative  assessment  tool  in  undergraduate  psychology  courses  found  that  students  who  used  the  tool 
achieved  higher  level  of  performance  than  those  who  did  not  (Buchanan,  2000).  Another  experimental  study  found 
that  improving  formative  assessment  could  actually  benefit  students’  performance  in  mandated  summative 
assessment  (Wiliam  et  al.,  2004). 


Using  Computers  in  Formative  Assessment 

Hunt  and  Pellegrino  (2002)  present  a couple  of  reasons  why  formative  assessment  has  not  been  a common  practice 
in  the  classroom.  First,  it  is  not  feasible  to  expect  all  teachers  to  be  experts  in  formative  assessment,  because  to 
perform  effective  formative  assessment,  teachers  need  to  know  ahead  of  time  all  the  materials  students  need  and  all 
the  potential  problems  students  may  have.  Second,  formative  assessment  is  very  time  consuming  because  it  requires 
teachers  to  identify,  analyze,  and  address  the  problems  each  individual  student  has.  It  is  not  realistic  to  ask  teachers 
to  spend  extensive  time  on  each  student.  Further,  formative  assessment  in  problem-based  learning  (PBL) 
environments  is  particularly  challenging  (Hoffman  & Ritchie,  1997).  These  environments  are  usually  open-ended. 
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and  students  take  initiative  in  negotiating  learning  objectives  and  assessment  methods.  Teacher  feedback  is  critical 
to  student  success,  but  the  ill-structured  nature  of  learning  environment  makes  formative  assessment  particularly 
difficult.  To  address  these  issues.  Hunt  and  Pellegrino  (2002)  propose  that  technology  tools  be  designed  to  automate 
formative  assessment  tasks.  This  agenda  is  supported  by  National  Research  Council  (NRC),  which  recommends  that 
developers  of  educational  materials  create  tools  “that  will  enable  teachers  to  implement  high-quality  instructional 
and  assessment  practices”  (NRC,  2001b). 

Researchers  usually  categorize  computer-supported  formative  assessment  based  on  the  content  of  feedback  provided 
to  the  learner.  McKendree  (cited  in  Merrill,  Reiser,  Ranney,  & Trafton,  1992)  distinguishes  between  feedback  that 
simply  points  out  that  an  error  has  occurred  from  those  that  also  remind  the  learner  of  the  current  goal  and  aspects  of 
the  error.  Similarly,  Reiser,  Connelly,  Ranney,  and  Ritter  (cited  in  Merrill  et  ah,  1992)  discusses  minimal  feedback 
strategies  which  only  inform  the  learner  of  the  error  without  further  information  as  compared  to  explanatory 
feedback  which  identify  the  aspects  of  the  solution  that  are  incorrect.  Narciss  (1999)  uses  the  level  of 
informativeness  to  classify  computer  provided  feedback  into  three  categories:  feedback  with  low  informativeness 
provides  knowledge  about  results;  feedback  with  medium  informativeness  offers  knowledge  about  mistakes; 
feedback  with  high  informativeness  embeds  knowledge  on  how  to  proceed.  Graesser,  VanLehn,  Rose,  Jordan,  & 
Harter  (2001)  discuss  three  levels  of  feedback:  backchannel  feedback,  evaluative  pedagogical  feedback,  and 
corrective  feedback.  Backchannel  feedback  acknowledges  the  learner’s  input  by  having  a pedagogical  agent 
periodically  nodding  and  saying  uh-huh  while  the  learner  is  typing  important  words  into  a system.  Evaluative 
pedagogical  feedback  conveys  negative,  neutral  negative,  neutral  positive,  and  positive  evaluation  of  the  learner’s 
responses.  Corrective  feedback  provides  canned  responses  to  correct  common  bugs  and  misconceptions. 

Despite  the  different  categories  or  taxonomies  of  formative  assessment/feedback,  researchers  seem  to  generally 
agree  that  the  levels  of  formative  assessment/feedback  can  vary  depending  on  how  much  information  provided  to 
the  learner.  Lower  level  feedback  only  indicates  that  an  error  has  occurred;  the  location  of  the  error,  the  aspects  of 
the  error,  and  suggestions  on  how  to  correct  the  errors  are  provided  in  higher  level  feedback.  Research  shows  that 
higher  level  feedback  tends  to  result  in  better  learner  performance  (Merrill  et  ah,  1992)  and  motivation  (Narciss, 
1999)  than  lower  level  feedback. 


The  Design  of  the  Assessment  System 

A web-based  system  is  being  designed  to  manage  the  implementation  of  VEL.  The  task  presented  in  a VEL  is  ill- 
structured  and  anchored  in  an  opening  scenario  or  movie.  Each  VEL  contains  information  resources  in  the  form  of 
virtual  scientific  instruments  and  hypermedia  assets  which  students  use  as  a resource  for  conducting  inquiry. 
Additionally,  each  VEL  contains  scaffolds  which  aid  students  in  conducting  inquiry.  A web-based  system  contains 
tools  for  assessing  student  learning  and  resources  for  supporting  teachers  in  effective  integration  of  the  VELs  into 
the  classroom. 

The  assessment  system  interfaces  with  the  VELs.  Students’  responses  are  delivered  from  the  VELs  to  the  assessment 
system.  The  teacher  accesses  students’  responses  through  a web  interface  and  provides  individualized  feedback.  The 
feedback  is  sent  from  the  web-based  system  to  the  VEL  and  presented  to  students  in  the  virtual  environment. 

The  design  of  the  assessment  system  was  informed  by  our  previous  work  that  examines  how  teachers  assessed 
student  learning  and  their  reasons  for  these  assessment  practices  when  using  Alien  Rescue,  a computer-based, 
problem-based  learning  (PBL)  environment  (Pedersen  et  ah,  2007).  The  study  suggests  the  following  design 
guidelines  for  improving  Alien  Rescue  and  other  PBL  programs.  The  first  guideline  is  to  embed  multiple-choice 
questions  within  PBL  activities.  Although  this  practice  is  inconsistent  with  social  constructivist  learning  theories, 
teachers  need  multiple-choice  questions  because  of  high  stakes  testing.  The  second  guideline  is  to  provide 
modifications  and  scaffolds  within  the  PBL  program  that  teachers  can  assign  to  individual  students  based  on  special 
needs.  Some  teachers  were  concerned  that  PBL  is  too  challenging  for  some  students.  They  may  need  to  provide 
greater  structure  and  support  for  them.  The  next  two  guidelines  concern  educative  components,  information 
embedded  in  the  PBL  program  to  educate  teachers  about  how  to  best  implement  the  program.  For  example, 
educative  components  should  present  research  findings  on  motivation,  especially  the  impact  of  grade  on  intrinsic 
motivation.  They  should  also  emphasize  the  value  of  student  reflection  on  their  process  and  solution  through  self 
and  peer  assessment. 
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Performance-centered  design  (Raybould,  1995)  also  provides  guidance  for  our  development  effort.  It  is  a system 
design  methodology  that  focuses  on  the  system  user  as  a task  performer  who  is  supported  by  a knowledge  base 
provided  in  a just-in-time  fashion.  It  aims  to  reduce  training  and  improve  task  performance  via  just-in-time 
information  and  easy-to-use  tools.  It  is  consistent  with  the  suggestion  to  embed  educative  components  in  PBL 
programs  for  teachers  to  implement  the  programs  as  they  were  designed  (Pedersen  et  ah,  2007).  Some  of  the  key 
design  strategies  include:  focusing  on  tasks,  processes,  and  the  natural  flow  of  work,  supporting  performance 
through  best  practice  processes,  establishing  or  aiding  in  establishing  goals,  providing  layered  access  to  resources 
(from  easy-to-understand  labels  to  information  boxes  to  linked  resources  and  training  programs). 

Assessment  design  within  each  VEL  is  informed  by  the  assessment  triangle  presented  in  Pellegrino,  Chudowky,  and 
Glaser  (2001).  To  assess  student  learning,  three  key  components  should  be  considered:  cognition,  observations,  and 
interpretations.  During  design  we  identify  the  aspects  of  student  cognition  that  are  important  to  measure.  Student 
artifacts  including  the  final  product  required  in  each  module  will  be  designed  to  allow  observation  of  this  cognition. 
Instruments  were  developed  to  support  teachers  in  interpreting  these  observations  in  terms  of  targeted  student 
competencies. 

Drawing  on  these  guidelines,  our  current  management  system  design  has  the  following  features:  task-oriented 
interface,  teacher  adjustable  scaffolds,  teacher  adjustable  assessments,  contextual  formative  assessment  support,  and 
educative  components 

Task  oriented  interface.  The  web-based  assessment  system  has  been  developed  to  support  teachers  in  implementing 
VELs  in  the  classroom.  We  identified  four  key  tasks  or  goals  that  teachers  may  need  to  complete  in  order  to  conduct 
formative  assessment  in  these  types  of  learning  environments:  configuring  module,  managing  students,  monitoring 
progress,  and  assessing  artifacts.  The  first  two  tasks,  configuring  modules  and  managing  student,  are  prerequisite  for 
formative  assessment.  They  involve  management  activities  such  as  setting  up  the  module  and  creating  student 
accounts.  It  allows  a teacher  to  set  the  game  access  dates  and  times,  to  determine  the  difficulty  level  for  an  entire 
class  or  for  individual  students,  to  choose  grading  methods,  to  modify  grading  rubrics,  and  to  identify  the  type  and 
amount  of  scaffolds  provided  for  an  entire  class  or  individual  students.  Student  information  will  be  entered  into  the 
system  before  students  can  enter  a VEL.  Teachers  may  also  edit  students’  information,  delete  or  add  students.  The 
next  two  tasks,  monitoring  progress  and  assessing  artifacts,  are  the  core  features  of  the  system.  They  support  daily 
monitoring  and  assessment  activities.  Monitoring  progress  feature  assists  teachers  when  they  monitor  student 
progress  in  the  VEL  without  any  specific  goal  of  assessing  certain  artifacts.  Teachers  may  start  by  viewing  a list  of 
student  names  and  the  current  tasks  that  they  are  completing.  Teachers  may  choose  a student  to  get  more  details 
such  as  current  task  status  and  performance,  completed  and  pending  activities  and  the  associated  evaluation,  time 
spent  in  the  game,  tools  used,  and  a list  of  game  sessions.  Assessing  artifacts  feature  enables  teachers  to  assess 
certain  artifacts  students  created  in  the  game.  From  a list  of  student  names  and  a list  of  artifacts,  teachers  may 
choose  to  grade  a certain  artifact  for  a certain  student.  Depending  on  the  artifact  type  and  grading  method  configured 
for  the  artifact,  appropriate  artifacts,  rubrics,  and  grading  suggestions  will  appear. 

Teacher  adjustable  scaffolds.  The  management  system  enables  teachers  to  adjust  the  support  provided  to  children 
as  they  work  in  the  VELs.  These  supports  can  be  assigned  by  the  teacher  to  a class,  group,  or  individual  student. 

For  example,  textual  content  embedded  in  a VEL  could  be  presented  at  a fourth  or  eighth  grade  reading  level.  The 
teacher  may  decide  to  change  this  scaffold  for  a class,  group,  or  individual  student. 

Teacher  adjustable  assessments.  Different  types  of  assessment  tools  (i.e.  rubric,  checklist,  numeric  score)  are 
available  for  the  teacher  to  edit  or  disable.  For  each  VEL,  there  are  various  artifacts  or  products  created  by  the 
children  as  they  play.  These  can  then  be  evaluated  using  assessments.  The  teacher  can  decide  which  artifacts  to 
evaluate.  She  may  also  decide  to  adjust  the  points  allocated  to  each  level  of  a rubric. 

Contextual  formative  assessment  support.  Contextual  support  is  provided  to  aid  in  formative  evaluation.  Examples 
include  student  exemplars,  scoring  guides,  and  learning  task  background  information.  We  adopted  several  design 
strategies  to  support  teachers’  use  of  the  system.  First,  we  highlighted  the  four  main  tasks  described  in  the  previous 
sections  by  creating  a tab  for  each.  We  hope  this  design  will  clarify  the  purpose  of  the  system  for  teachers  and  guide 
them  in  using  the  system.  Second,  we  dedicated  an  information  box  on  the  screen  to  embed  educative  components 
that  are  relevant  to  the  context  in  which  the  teacher  is  working.  For  example,  as  discussed  in  the  previous  section, 
we  provide  examples  to  guide  teachers  in  providing  feedback.  This  is  an  important  feature,  because  teachers  who  are 
new  to  the  game  may  not  know  what  to  expect  from  students  and  they  may  not  know  how  to  provide  feedback  so 
that  it  seems  to  come  from  the  virtual  mentor. 

Educative  components.  This  aspect  of  our  design  still  needs  additional  work  before  it  is  fully  implemented.  The 
educative  components  will  provide  information  to  the  teacher  to  facilitate  a better  understanding  of  the  theoretical 
underpinnings  and  methods  for  successful  implementation  of  the  VELs. 
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Abstract 

Spatial  presence  is  an  integral  part  of  virtual  learning  environments  (VLE’s).  Research  findings  indicate 
that  spatial  presence  in  VLE's  may  positively  affect  learning  by  stimulating  abstract  thinking  and  problem  solving 
capabilities.  This  paper  proposes  research  into  the  two  principle  components  of  spatial  presence,  the  style  of  the 
visual  display  and  the  organization  of  interactive  elements,  to  better  understand  design  elements  that  make  this 
emerging  medium  instructionally  effective. 

Introduction 

Virtual  reality  (VR)  is  an  engaging  and  attractive  form  of  multimedia  computing  that  is  gaining  a strong 
foothold  in  our  daily  lives.  VR's  unique  array  of  affordances  and  opportunities  for  interactivity  allow  for  a wider 
range  of  user  controls  than  other  forms  of  multimedia.  Use  of  virtual  reality  in  science,  engineering,  aeronautics,  and 
medicine  has  been  developing  since  1987  in  projects  funded  by  the  federal  government,  particularly  the  Department 
of  Defense,  the  National  Science  Foundation,  and  the  National  Aeronautics  and  Space  Administration  (Virtual 
reality,  2008).  Over  the  past  decade,  the  use  of  virtual  reality  in  entertainment  games  has  increased  dramatically. 
Currently,  scientific  research  based  on  computer  simulation,  called  e-Science,  is  a successful  method  used  in  the 
physical  and  natural  sciences,  medicine  and  engineering  for  experimental  scientific  discovery  (Getov,  2009). 

Educational  use  of  virtual  worlds  is  increasingly  rapidly  (Bainbridge,  2007;  Chittaro,  & Ranon,  2007). 
Contemporary  VLE’s  are  used  for  many  types  of  subject  matter  and  learning  outcomes,  including  medicine, 
accounting,  business,  art,  history,  science  and  math;  and  for  training  of  soft  skills  such  as  management  and 
negotiation.  Two  popular  multi-user  web  based  virtual  reality  programs  used  by  educators  are  Second  Life  and 
Active  Worlds. 

VR  has  been  described  as  a “partnership  between  computers  and  the  human  brain”  accomplished  through 
the  use  of  various  input  devices  (Virtual  reality,  2008).  The  experience  is  predominantly  visual,  but  can  include  the 
other  senses:  sound,  touch,  smell.  What  distinguishes  it  from  other  multimedia  computer  experiences  is  the  user’s 
sense  of  being  there,  in  the  virtual  world,  accomplished  through  interaction  with  a vivid,  immersive  simulated 
environment.  This  is  referred  to  in  the  literature  as  presence.  Presence  is  a state  of  mind,  the  user’s  sense  of  being 
present  in  a virtual  space  (Ijsselsteijn,  2003;  Lombard  & Ditton,  1997;  Winn,  1993).  Presence  may  include  a spatial 
sense  of  being  in  the  virtual  environment,  and/or  a social  sense  of  being  in  a space  with  other  sentient  beings 
(Spagnolli,  Varotto,  & Mantovani,  2003).  Social  presence  can  occur  in  text  chats  or  teleconferences  without 
graphics  or  simulations  (Jacobsen,  2001),  but  spatial  presence  requires  a visual  display  (Schubert,  Friedmann,  & 
Regenbrecht,  2001). 

Spatial  presence  is  a theoretical  construct  that  can  guide  design  and  development  of  virtual  learning 
environments.  Effective  VLE’s  will  utilize  the  affordance  of  spatial  presence  through  carefully  designed  visual 


487 


displays  and  user  interactivity.  This  reflection  paper  reviews  current  scholarly  literature  into  the  nature  of  spatial 
presence  in  VLE’s,  and  how  spatial  presence  may  affect  and  impact  learning  outcomes.  Based  on  conclusions  from 
this  literature  review,  the  authors  describe  and  propose  new  research  into  the  interactive  component  of  spatial 
presence. 

Literature  selected  for  this  review  consists  of  peer  reviewed  articles  from  scholarly  journals  in  the  fields  of 
education,  psychology  and  human-  computer  interaction.  This  includes  publications  devoted  exclusively  to  the 
study  of  presence,  such  as  Presence-Teleoperators  and  Virtual  Environments  (published  by  MIT  press  journals)  and 
the  International  Society  of  Presence  Research  website,  and  other  publications  whose  focus  is  educational 
technology  and  research.  Relevant  research  into  the  effects  of  spatial  presence  in  formal  education  or  training 
settings,  with  adult  or  high  school-age  populations,  is  examined  for  conclusions  and  need  for  further  research. 
Studies  that  use  children  as  participants  are  excluded  unless  adult  populations  are  included  in  the  same  study. 

Studies  that  use  VLE’s  to  examine  or  treat  psychological  disorders,  such  as  phobias  or  hallucinations,  are  also 
excluded  from  this  report.  Social  presence  is  discussed  only  as  it  affects  the  user’s  sense  of  spatial  presence. 

Defining  Spatial  Presence 

Spatial  presence  includes  realism,  transportation  (the  ability  to  move  to  different  spaces,  or  bring  objects 
closer  to  the  viewer)  and  immersion  or  user  involvement.  The  user’s  perception  of  their  potential  for  action  in  the 
space  and  the  resulting  consequences  are  important  dynamics  in  the  experience  of  spatial  presence  (Chertoff,  & 
Schatz,  2008;  Mantovani,  & Castelnuovo,  1999;  Spagnolli,  Varotto,  & Mantovani,  2003).  Spatial  presence  defines 
virtual  reality  and  separates  it  from  other  multimedia  programs  (Mikropoulous,  2006;  Romano,  & Brna,  2001; 
Stanney,  2002). 

Spatial  presence  fits  the  definition  of  a perceived  affordance  as  described  by  Norman  (2004)  because  it 
affords  the  user  a unique  perception  of  potential  interaction  with  a simulated  environment.  In  a non-virtual  interface, 
the  user  perceives  icons  and  text  links  as  having  the  potential  for  displaying  different  web  pages,  windows,  or 
activating  specific  programs.  In  virtual  reality,  the  user  perceives  the  interface  as  an  interactive  spatial  environment 
in  which  they  have  a presence  that  is  separate  or  in  addition  to  their  existence  in  the  real  world.  Their  actions  in  the 
virtual  world  can  affect  the  virtual  environment  itself. 

Sensory  stimuli  make  spatial  presence  an  external  as  well  as  an  internal  state  (Marsh,  2003;  Waterworth,  & 
Waterworth,  2001).  Virtual  reality  can  include  audio  and  haptic  sensations,  but  it  is  primarily  the  visual  display  and 
user  interactivity  that  creates  the  sense  of  being  in  a virtual  space  (Schubert,  Lriedmann,  & Regenbrecht,  2001).  The 
virtual  spatial  experience  blends  the  learner’s  internal  visualization  with  the  computer  generated  one. 

Spatial  presence  is  sometimes  associated  with  the  theory  of  embodied  cognition  (Wilson,  2002)  because  it 
is  characterized  by  a sensation  of  a strong  perception-action  link  between  the  display  and  the  user  (Schubert, 
Lriedmann,  & Regenbrecht,  2001).  Presence  includes  the  user’s  conscious  suspension  of  disbelief,  a willingness  to 
immerse  oneself  in  the  simulated  environment  and  to  block  out  the  real  environment  (ISPR,  2000).  This  results  in 
the  user’s  sense  of  being  in  two  worlds  simultaneously,  the  real  and  the  virtual,  in  a sort  of  blended  reality. 

The  International  Society  for  Presence  Research  describes  presence  as 

“A  psychological  state  or  subjective  perception...  of  an  individual's  experience...  filtered  through 
human  made  technology...  part  or  all  of  the  individual's  perception  fails  to  accurately  acknowledge  the  role 
of  technology  in  the  experience. . . To  some  degree  the  environments  are  experienced  as  if  the  technology 
was  not  involved”  (2000). 

Virtual  Learning  Environments  in  Education 

The  convergence  of  presence  with  constructivist  learning  theories  became  of  particular  interest  with  the 
development  of  computer  generated  three  dimensional  environments  that  could  create  first  person  experiences  in  a 
simulated  environment  (Winn,  1993).  In  early  research  with  virtual  reality  simulations  during  the  1990’s,  Winn 
pointed  out  that  the  users’  sense  of  interaction  with  an  environment  stimulates  deep  learning.  The  immersive 
environment  eliminates  the  need  for  a symbol  system,  and  enables  direct  first  person  learning  experiences  that  are 
“...direct,  personal,  subjective  and  often  tacit...”  in  a safe  and  controlled  environment  (1993).  This  fits  well  with 
constructivist  learning  theories  of  knowledge  construction.  That  is,  that  knowledge  construction  occurs  most 
efficiently  when  learners  are  actively  involved  in  activities  that  are  personally  relevant  (Chittaro,  & Ranon,  2007). 
Subsequent  research  indicates  that  spatial  presence  is  important  in  virtual  worlds  to  accomplish  learning  goals 
(Jacobson,  2001)  and  has  positive  effects  on  students'  perceptions  of  the  course  communications  and  relevance 
(Nishide,  Shirna,  Araie,  & Ueshima,  2007;  Reznick,  & MacRae,  2006;  Takatalo,  Nyman,  & Laaksonen,  2008). 
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Spatial  Presence 

Spatial  presence  in  VLE's  is  generally  considered  to  benefit  learning  by  creating  first  person  experiences, 
as  in  simulations,  and  by  enabling  observation  of  processes  or  phenomena  that  would  be  difficult,  expensive  or 
impossible  to  observe  in  the  real  world.  But  some  of  the  most  intriguing  observations  from  research  into  spatial 
presence  are  its  effects  on  higher  thought  processes:  conceptual  understanding  and  creating  mental  models  (Chittaro 
& Ranon,  2007;  Kontogeorgiou,  & Bellou,2008;  Limnou  & Roberts,  2008;  Zacharia,  2007),  problem-solving  (Cai, 
Lu,  Zheng,  & Li,  2005;  McClean  & Saini-Eidukat , 2001),  metacognitive  and  abstract  thinking  (Antonietti  & 
Cantoia,  2000;  Cai  et  ah,  2005). 

In  virtual  reality,  the  user  perceives  the  interface  as  an  interactive  spatial  environment  in  which  they  have  a 
presence  that  is  separate  or  in  addition  to  their  existence  in  the  real  world.  The  learner’s  awareness  of  being  in  the 
two  environments  simultaneously  may  stimulate  metacognitive  reflection.  The  experience  of  presence  that  occurs  in 
VLE’s  can  connect  learners’  imagination  and  internal  mental  models  with  an  external  guided  stimulus  to  help  them 
“...find  points  of  convergence  and  divergence  between  different  worlds  to  understand  their  relationships  to  one 
another...”  (Thomas  & Brown,  2007,  p.150). 

The  interactive  element  of  spatial  presence  is  important  in  helping  users  build  mental  models  and  connect 
them  to  existing  knowledge.  Learners  interpret  their  experiences  and  use  them  to  refine  their  mental  models  (Dede, 
1995).  Mental  models  help  the  learner  visualize  the  phenomena  and  construct  analogies  (Seel,  2006).  The  reciprocal 
nature  of  learning  and  doing  in  VLE’s  can  build  conceptual  understanding  (Barab,  Hay,  Barnett,  & Squire,  2001). 

Constructivist  Learning  Theories  and  VLE’s 

Most  educational  programs  in  virtual  worlds  are  designed  around  constructivist  learning  theories,  that  is, 
that  deep  learning  is  achieved  through  active  student  involvement  in  the  learning  process.  Two  constructivist 
learning  theories  that  might  help  guide  design  and  development  of  VLE's  to  include  spatial  presence  as  an 
affordance,  are  multimedia  learning  theory  (MLT;  Mayer,  2001)  and  cognitive  flexibility  theory  (CFT;  Spiro, 
Coulson,  Feltovich,  & Anderson,  1992).  Both  MLT  and  CFT  consider  how  student  involvement  with  media  might 
enhance  or  decrease  deep  learning.  MLT  applies  Sweller’s  cognitive  load  theory  to  learning  from  words  and 
pictures  and  recommends  arrangement  of  text  and  media  that  will  result  in  germane  cognitive  load.  CFT  advocates 
richness  of  content  in  a flexible,  open  environment,  with  multiple  representations  at  different  levels  of  instruction 
(Spiro,  Collins,  & Ramchandran,  2007).  CFT  emphasizes  unstructured  learner  control  of  content  selection  and  pace 
to  encourage  flexible  thinking  by  the  learner,  whereas  MLT  proposes  a more  structured  program  control  of  media. 
Both  theories  aim  for  what  Sweller  calls  germane  cognitive  load,  the  optimum  effort  required  for  deep  learning 
(Clark,  Nguyen,  & Sweller,  2006). 

Design  of  VLE’s 

How  should  virtual  reality  interactive  displays  be  designed  to  develop  flexible  habits  of  mind,  and  help  the 
learner  integrate  new  knowledge  with  existing  knowledge?  How  does  style  and  manner  of  presentation  of  the  visual 
display  and  interactive  elements  affect  and  impact  learning  outcomes?  The  two  most  significant  components  of 
spatial  presence  are  the  visual  display  and  user  interactivity  (Lessiter,  Freeman,  Keogh,  & Davidoff,  2001;  Schubert, 
Friedmann,  & Regenbrecht,  2001). 

Visual  Design 

Effective  visual  design  of  VLE’s  is  governed  by  the  same  fundamental  principles  used  in  visual  art,  graphic 
design,  animation  and  cinema,  grounded  in  psychological  theories  of  perception.  Traditional  visual  media  create 
memorable  experiences  and  direct  the  viewers’  attention  to  key  elements  through  judicious  combinations  of  the 
scientific  principles  of  optics,  accepted  practices  of  illustrating  spatial  depth,  and  psychological  theories  of 
perception  and  the  viewer’s  organization  of  images.  Theoretical  foundations  include  Gestalt  theories  of  visual 
perception,  Mayer’s  multimedia  theory,  Sweller’s  cognitive  load  theory,  Paivio’s  dual  coding  theory 

Visual  design  principles  and  theories  can  be  applied  to  the  visual  design  of  graphic  elements  in  VLE’s  to 
enhance  the  users’  sense  of  spatial  presence  and  reduce  extraneous  cognitive  load.  In  VLE’s,  as  in  other  types  of 
visual  design,  adherence  to  real  world  imagery  that  is  not  relevant  to  the  learning  outcome  may  create  extraneous 
cognitive  load.  Luo,  & Duh  (2009)  concluded  that  detailed  architectural  and  landscape  features  in  VLE’s  can  cause 
slower  learner  reaction  times  and  reduced  memory.  Circular  environments  seem  to  be  easier  to  navigate  and 
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remember  than  square  environments,  possibly  because  the  weaker  ecological  features  of  cylindrical  rooms  (fewer 
angles  to  perceive  and  navigate)  reduce  extraneous  cognitive  load  (Luo  & Duh,  2009;  Jansen-Osmann, 

Wiedenbauer,  Schmid,  & Heil,  2007). 

Interaction  Design 

Research  findings  indicate  that  the  users’  interaction  or  sense  of  potential  for  interaction,  in  the  virtual 
world  is  the  most  significant  factor  in  the  experience  of  spatial  presence  (Chertoff,  & Schatz  2008;  Lessiter, 
Freeman,  Keogh,  & Davidoff,  2001;  Mantovani,  & Castelnuouvo,  1999;  Schubert,  Friedmann,  & Regenbrecht, 

2001;  Spagnolli,  Varotto,  & Mantovani,  2003).  Students’  interaction  with  computer  mediated  technologies  can 
combine  for  powerful  teaching  and  learning  (Cifuentes,  Carpenter,  & Bulu,  2008).  A meta-analysis  of  studies  of 
distributed  learning  environments  that  have  analyzed  subjective  vs.  objective  control  concludes  that  any  type  of 
learner  control  enhances  learning  (Kraeger  & Jerden,  2007).  Spatial  presence  is  a metacognitive  awareness  of  the 
potential  for  action  in  both  the  real  world  and  the  virtual  world.  More  research  is  needed  to  develop  models  for  the 
design  of  learner  interaction  in  VLE’s  that  will  optimize  learning  outcomes  and  minimize  extraneous  cognitive  load. 
User  control  features  such  as  free  navigation,  first-person  point  of  view,  manipulation  of  size  and  reification  of 
concepts  should  be  studied  for  their  effects  on  learning. 

Proposed  Research  into  Components  of  Spatial  Presence 

Merrill  refers  to  instructional  strategies  as  “...algorithms  for  manipulating  data  (knowledge  objects)...”  His 
instructional  transaction  theory  proposes  that  patterns  of  learner  interactions  can  be  designed  to  teach  specific  skills 
and  knowledge  (1999).  The  proposed  research  will  test  two  possible  algorithms  for  learner  interaction  in  a VLE  to 
teach  visual  design  concepts.  In  the  high  interaction  tutorial,  the  knowledge  objects  will  be  interactive  graphic 
elements  that  learners  will  be  required  to  manipulate.  The  high  interaction  tutorial  will  combine  what  Cifuentes  calls 
. .learner  interaction  with  subject  matter. . .”  and  . .learner  interaction  with  technology. . .”  (personal 
communication  September  16,  2009)  to  the  degree  that  is  practical  and  reasonable,  making  the  technology  a 
reification  of  the  subject  matter.  The  design  of  the  virtual  space  will  maximize  learner  interaction  with  reified 
subject  matter,  and  minimize  extraneous  interactions  with  technology.  In  the  low  interaction  tutorial  learners  will 
interact  with  generic  graphic  elements  to  display  non-interactive  knowledge  objects  that  they  will  view  passively. 

Research  Questions 

Does  use  of  increased  interactivity  in  the  form  of  controllable,  reified  subject  matter  in  a VLE  improve 
transfer  of  learning?  How  does  it  help  learners  build  mental  models  of  concepts?  Does  it  increase  the  learners’  sense 
of  spatial  presence?  Does  increased  interaction  and/or  spatial  presence  affect  the  mechanisms  associated  with 
constructivist  learning  and  increase  critical  thinking? 

Research  Design 

For  the  pilot  study  the  research  will  be  qualitative,  due  to  the  small  number  of  participants.  Participants 
will  be  students  enrolled  in  a graduate  education  course.  Two  different  tutorial  environments  will  be  designed, 
each  with  different  degrees  of  user  control  and  interaction.  The  graphics  and  text  in  each  tutorial  space  will 
contain  the  same  information.  Half  the  class  will  start  with  the  high  interaction  tutorial;  the  other  half  will  start 
with  the  low  interaction  version.  After  they  complete  each  tutorial,  students  will  be  given  a text  instructional 
message  and  be  asked  to  create  a graphic  illustration  of  the  message  that  emphasizes  the  important  concepts. 
They  will  then  repeat  the  process  using  the  other  version  of  the  tutorial,  and  completing  a different  post-  tutorial 
assignment.  The  interactive  tutorial  space  is  shown  in  Figure  1 . 
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Figure  1.  The  high  interaction  tutorial  will  be  unguided,  to  intensify  the  users’  perception  of  their  potential 
for  action  in  the  space 

Participants  will  answer  questions  from  the  Witmer  and  Singer  Presence  Questionnaire  following  their  use 
of  each  of  the  tutorials  and  before  completion  of  each  design  assignment.  Their  completed  design  assignments  will 
be  evaluated  by  two  or  three  experts  in  instructional  visual  design.  Participants  will  write  a paragraph  describing  the 
process  they  used  in  creating  their  design  after  they  complete  each  design  assignment.  These  reports  will  be 
analyzed  using  qualitative  coding  analysis  methods,  for  evidence  of  critical  thinking,  such  as  the  frequency  of  “how” 
or  “why”  statements  or  questions  (high  critical  thinking),  compared  to  “what”  statements  or  questions  (low  critical 
thinking).  They  will  also  complete  post-treatment  questionnaires  that  measure  the  occurrence  of  mechanisms 
associated  with  constructivist  learning  and  their  perceptions  of  the  effectiveness  of  the  tutorial  for  transfer  of 
learning. 

Participants  will  receive  detailed  instructions  for  joining  Second  Life,  obtaining  an  avatar,  and  practicing 
basic  skills  in  Second  Life  about  two  weeks  before  starting  the  assignment.  They  will  complete  these  activities  on 
their  own  schedule  before  beginning  the  exercise.  When  they  begin  the  exercise,  they  will  receive  instructions  for 
using  each  tutorial.  The  course  requires  use  of  image  editing  software,  so  it  can  be  assumed  that  they  have  sufficient 
technical  skills  to  complete  the  design  assignments. 


491 


Measures 


Measure  One 

Does  increased  interactivity  increase  learners’  sense  of  spatial  presence  while  using  the  tutorial? 

From  Witmer,  & Singer’s  1998  presence  questionnaire: 

1 . How  much  were  you  able  to  control  events? 

2.  How  responsive  was  the  environment  to  actions  that  you  initiated  (or  performed)? 

3.  How  natural  did  your  interactions  with  the  environment  seem? 

4.  How  much  did  the  visual  aspects  of  the  environment  involve  you? 

5.  How  natural  was  the  mechanism  which  controlled  movement  through  the  environment? 

6.  How  aware  were  you  of  events  occurring  in  the  real  world  around  you? 

7.  How  aware  were  you  of  your  display  and  control  devices? 

8.  How  compelling  was  your  sense  of  objects  moving  through  the  environment? 

9.  How  completely  were  you  able  to  actively  survey  or  search  the  environment? 

10.  How  compelling  was  your  sense  of  moving  around  inside  the  virtual  environment? 

11.  How  well  could  you  examine  objects  from  multiple  viewpoints? 

12.  How  well  could  you  move  or  manipulate  objects  in  the  virtual  environment? 

13.  To  what  degree  did  you  feel  confused  or  disoriented  at  the  beginning  of  the  tutorial? 

14.  How  involved  were  you  in  the  virtual  environment  experience? 

15.  How  distracting  were  the  control  mechanisms? 

16.  How  quickly  did  you  adjust  to  the  virtual  environment  experience? 

17.  How  proficient  in  moving  and  interacting  with  the  virtual  environment  did  you  feel  at  the 
end  of  the  experience? 

18.  How  well  could  you  concentrate  on  the  assigned  tasks  or  required  activities? 

19.  Were  you  involved  in  the  task  to  the  extent  that  you  lost  track  of  time? 


Measure  Two 

Does  use  of  increased  interactivity  in  the  form  of  controllable  reifications  of  the  subject  matter  in  a VLE 
improve  transfer  of  learning?  How  well  do  students  understand  the  concepts  presented  in  the  tutorial  and  transfer 
them  to  novel  design  problems? 

Post  treatment  questionnaire  item  stems: 

1.  The  concepts  were  presented  clearly  in  the  tutorial. 

2.  I was  able  to  understand  the  topics  presented  in  the  context  of  instructional  design 

3.  I used  the  concepts  presented  in  my  design  assignment. 

4.  I will  apply  what  I learned  in  future  tasks 

5.  I learned. 

Following  their  use  of  the  tutorial,  the  participants  will  create  a visual  instructional  message  based  on  an 
assigned  text  message.  Their  completed  designs  will  be  assessed  by  two  to  three  raters,  using  the  rubric  shown  in 
Table  1. 
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Table  1. 

Rubric  for  Design  Assignment. 


Grading  Criteria 

Maximum  Points 

The  design  utilizes  the  shape  and  level  of  detail  of  graphic 
components  to  place  emphasis  on  one  to  three  crucial  concepts  of  the 
message,  de-emphasize  less  important  elements,  and  create  an  overall 
visual  unity. 

10 

The  design  utilizes  the  size  of  graphic  components  to  place 
emphasis  on  one  to  three  crucial  concepts  of  the  message,  de-emphasize 
less  important  elements,  and  create  an  overall  visual  unity. 

10 

The  design  utilizes  position  of  graphic  components  to  place 
emphasis  on  one  to  three  crucial  concepts  of  the  message,  de-emphasize 
less  important  elements,  and  create  an  overall  visual  unity. 

10 

The  design  utilizes  color  and/or  brightness  of  graphic 
components  to  place  emphasis  on  one  to  three  crucial  concepts  of  the 
message,  de-emphasize  less  important  elements,  and  create  an  overall 
visual  unity. 

10 

The  designs  utilizes  texture  or  pattern,  of  graphic  components  to 
place  emphasis  on  one  to  three  crucial  concepts  of  the  message,  de- 
emphasize  less  important  elements,  and  create  an  overall  visual  unity. 

10 

The  design  uses  proximity,  common  fate  and  variance  to 

connect  related  elements  and  create  unity. 

10 

Style  and  detail  of  imagery  is  appropriate  to  the  message. 

10 

Selected  imagery  promotes  a conceptually  clear  and  unified 

message. 

10 

Written  explanation  of  design  process  describes  impact  of  Second 
Life  tutorial  on  transfer  of  concepts  to  finished  design. 

20 

Total 

100 

Measure  Three 

Does  the  design  facilitate  the  mechanisms  associated  with  constructivist  learning  as  measured  by  learners’ 
self  reports?  Does  the  design  engage  and  challenge  learners,  motivate  learners  to  take  responsibility  for  learning,  and 
foster  active  social  discovery  of  meaning  and  dynamic  interactions  between  learners  and  the  environment? 

Post  treatment  questionnaire  item  stems: 

1 . Experience  in  the  environment  helped  me  develop  and  refine  my  ideas  for  the  subsequent 
assignment. 

2. 1 understood  the  concepts  that  were  presented  in  the  environment. 

3.  The  learning  goals  were  clear  to  me. 

4.  I felt  like  an  active  member  in  the  environment. 

5. 1 actively  explored  the  principles  and  concepts  presented 

6. 1 felt  that  I had  control  to  explore  the  environment  in  a manner  that  interested  me. 

7.  I was  able  to  create  a valuable  learning  experience  during  the  time  I spent  in  the  VLE 

8.  The  environment  stimulated  my  interest  in  learning  the  concepts 

9.  The  subsequent  assignments  were  challenging. 

Data  Coding  and  Entry 

Student  names  will  be  replaced  by  a code  number  for  the  reports.  Questionnaires  will  be  conducted 
anonymously  on  the  internet,  with  encryption  used  to  protect  their  identity.  Questionnaires  will  not  be  matched  to 
self  reports. 
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Analyses 

Students’  design  projects  will  be  assessed  by  3 experts  in  instructional  visual  design  using  a common 
rubric.  Students’  responses  to  interview  questions  and  written  assignment  will  be  analyzed  using  qualitative  coding 
and  analysis  methods.  Results  from  this  pilot  study  will  be  used  to  design  a larger,  more  detailed  study  involving  a 
sufficient  number  of  participants  to  use  quantitative  methods  of  analysis  of  learning  outcomes  and  sense  of  spatial 
presence. 
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Abstract 

Expert  instructional  designers  make  sense  of  ill-structured  problem  situations  more  quickly  than  novices  by  drawing 
upon  prior  knowledge,  previous  experiences,  and  an  “established  repertoire  of  heuristics''  (Day  & Lord,  1992,  p. 
45).  Heuristics,  or  rules  of  thumb,  are  guidelines  that  enable  designers  to  respond  to  complex  problems  more 
efficiently.  This  study  examined  heuristics  used  by  expert  instructional  designers  when  engaged  in  the  instructional 
design  (ID)  process.  Qualitative  interviews  were  used  to  capture  the  heuristics  used  by  16  instructional  designers 
when  completing  complex  ID  projects  and  resulted  in  25  ID  heuristics,  divided  into  eight  categories.  Although  some 
of  the  identified  design  heuristics  overlap  with  other  fields  of  study  such  as  business  management  and 
communication,  others  are  unique  to  this  study.  Implications  for  the  education  of  instructional  designers  are 
discussed. 

Extracting  Heuristics  from  Expert  Instructional  Designers 

Solving  problems  is  a big  component  of  what  instructional  designers  do  (Muraida  & Spector,  1993;  Perez 
& Emery,  1995;  Rowland,  1991;  Seel,  Eichenwald.  & Penterman,  1995).  Typically,  designers  are  presented  with  an 
ill-structured  situation  and  asked  to  close  the  gap  between  the  current  situation  and  the  intended  end  point.  This  can 
be  considered  a “problem."  When  solving  a problem,  a person  not  only  applies  rules,  but  he/she  combines  multiple 
rules  in  order  to  arrive  at  a solution  (Gagne,  1985).  The  solving  of  a newly  encountered  problem  could  require  a 
learner’s  use  of  declarative  knowledge  and  cognitive  strategies  as  well  as  the  inclusion  of  previously  learned 
relational  and  procedural  rules.  Johnson  (1988)  elaborated  on  the  use  of  rules  in  solving  ill-structured  problems 
when  he  stated,  “. . . no  single  correct  procedure  exists,  and  there  is  no  definitive  way  of  assessing  the  correctness  of 
a rule  based  upon  the  outcome  of  a single  case.  There  is  no  optimally  correct  rule. . . only  rules  which  are  relatively 
more  accurate”  (p.  212).  Ingram  (1988)  noted,  when  solving  ill-structured  problems,  there  is  no  one  set  of  strategies 
that  can  be  used  in  all  situations,  thus  many  people  rely  on  heuristics  to  solve  problems. 

This  study  is  about  heuristic  use  by  instructional  designers.  Heuristics  are  “simple  decision  rules  through 
which  individuals  make  judgments”  (Dudczak,  1995,  p.  4).  Ingram  (1988)  defined  a heuristic  as  “a  general  strategy 
for  approaching  a problem”  (p.  215).  Heuristics  help  to  simplify  the  problem  (Abel,  2003).  Heuristics  may  be  called 
many  things:  informal  procedures,  generalizations,  strategies,  principles,  decision  rules,  intuition,  or  rules  of  thumb 
[list  not  inclusive].  For  purposes  of  this  paper,  ‘heuristics'  and  ‘rules  of  thumb’  will  be  used  interchangeably  (Abel, 
2003). 

The  use  of  heuristics,  by  experts,  has  been  examined  by  a number  of  researchers  (e.g..  Day  & Lord.  1992; 
Murphy,  2005;  Seel  et  al.,  1995;  Woods,  1988).  In  a study  by  Day  and  Lord  (1992),  38  experts  and  30  novices 
participated  in  the  task  of  sorting  organizational  problems.  Results  showed  that  experts  categorized  the  ill-structured 
problems  faster  than  novices.  They  specifically  stated,  “Experienced  executives  quickly  make  sense  of  relevant 
problems  and  draw  upon  an  established  repertoire  of  heuristics  to  guide  their  sense-making  efficiently”  (p.  45).  Chi 
(2006)  stated  that  experts  may  use  heuristics  of  which  novices  are  not  aware.  However,  experts  may  perform  with 
some  heuristics  so  automatically,  that  even  the  expert  is  unaware  of  their  use  (Lee  & Reigeluth,  2003). 

One  of  the  pioneering  efforts  regarding  heuristics  in  instructional  design  is  a piece  by  Haney,  Lange,  and 
Barson  (1968).  The  premise  was  to  provide  instructional  developers  with  heuristics  to  use  when  working  with 
professors  in  higher  education.  From  empirical  observations  of  an  instructional  development  project,  they  created  a 
collection  of  18  heuristics.  These  heuristics  are  so  general  that  even  though  they  were  compiled  40  years  ago,  they 
still  apply  today.  For  example,  some  of  their  heuristics  included:  “Always  move  toward  determining  the  professor’s 
objectives,”  “Involve  the  student  in  the  developmental  process,”  “Stress  the  human  elements  in  an  instructional 
system,”  and  “Don't  let  the  words  get  in  the  way”  (pp.  4-5).  Hoban,  Heider,  and  Stoner  (1980)  elaborated  on  Haney 
et  al.  (1968)  by  identifying  eight  additional  heuristics  they  noted  from  their  empirical  observations  of  a large 
instructional  development  project.  Some  of  their  heuristics  included:  “Carefully  assess  the  need,”  “Do  not  try  to 
please  everyone,”  and  “Set  reasonable  time  limits”  (pp.  8-9).  Noel  and  Hewlett’s  (1981)  theoretical  article  on 
heuristics  in  instructional  development  derived  from  their  own  experiences  and  that  of  colleagues.  They  explained 
when  heuristics  can  be  used  in  the  development  process.  They  listed  both  general  heuristics  as  well  as  heuristics  that 
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align  with  the  instructional  systems  development  (ISD)  model.  For  example,  heuristics  included:  “Adopt  an  ISD 
model,  then  adapt  it,”  “Plan  on  how  long  it  will  take,  then  plan  on  it  taking  longer,”  “Test  the  objective,”  and  “Keep 
your  eyes  and  your  options  open”  (pp.  15-17). 

Ertmer  et  al.  (2008)  suggested  expert  instructional  designers  use  their  prior  knowledge  and  experiences  to 
create  and  apply  personal  “rules  of  thumb”  (i.e.,  heuristics)  when  solving  instructional  design  problems.  This  idea 
has  been  supported  by  other  researchers  (i.e.,  Haack  & Mischke,  2005;  Kolodner,  1997;  Rowland,  1993).  These 
rules  are  not  taken  from  textbooks,  but  are  much  more  personalized  - acquired  from  the  individual’s  schema  of 
previous  experiences  that  they  bring  to  the  design  situation  (Klein  & Calderwood,  1988;  Kolodner  & Guzdial, 

1999).  In  his  textbook.  Designing  Instructional  Systems , Romiszowski  (1981)  conjectured  that  instructional 
designers  take  the  systematic  approach  they  were  taught  and  change  it  into  heuristics  they  can  use. 

Purpose 

This  study  was  designed  to  explore  the  heuristics  experienced  instructional  designers  use  when  engaged  in 
the  ID  process.  In  order  to  achieve  this,  semi-structured  interviews  took  place  with  16  purposefully  selected 
instructional  designers  who  had  practiced  ID  for  at  least  ten  years.  During  the  interviews,  participants  described  their 
experiences  in  the  form  of  narratives,  reflecting  on  complex  or  challenging  ID  projects  they  had  worked  on.  The 
research  question  was:  What  heuristics  do  experienced  instructional  designers  use  to  solve  instructional  design 
problems  ? 

Theoretical  Framework 

A phenomenological  research  design,  utilizing  semi-structured  interviews  (Patton,  2002)  of  expert 
instructional  designers,  was  used.  Phenomenology  is  an  approach  that  revisits  the  conscious  experience  as  a way  of 
capturing  the  core  of  that  experience  through  thorough  first  person  descriptions  (Moustakas,  1994;  Smith,  2005).  In 
this  study,  interviews  elicited  this  first-person  point  of  view  and  situated  it  within  the  participants’  narratives  of  their 
instructional  design  experiences.  In  this  study,  the  researchers  elicited  multiple  participants’  personal  experiences  of 
instructional  design  in  order  to  get  an  accurate  picture  of  heuristics  used  in  actual  (not  theoretical)  instructional 
design  projects.  A component  of  phenomenology  allows  for  the  examination  of  the  “essence  of  shared  experience” 
(Patton,  2002,  p.  106).  Therefore,  phenomenology  was  the  approach  that  was  undertaken  for  this  research. 

Methodology 

Participants  and  Site 

A purposeful  sampling  technique  (Creswell,  2007)  was  used  to  select  participants  who  had  at  least  10  years 
of  instructional  design  experience,  were  currently  active  in  the  field,  and  were  known  by  their  peers  as  expert 
instructional  designers.  An  initial  list  of  potential  participants  (n  = 31)  was  created  from  a list  of  “seasoned” 
professionals  who  regularly  attended  the  annual  conference  of  the  Association  for  Educational  Communications  and 
Technology  (AECT).  In  addition,  participants  were  selected  based  upon  their  availability  for  either  a face-to-face 
interview  at  the  2007  AECT  conference  or  via  a phone  or  video  conference  in  the  three  months  following  AECT. 

The  snowball  method  (participants  identifying  other  potential  participants)  was  used  to  increase  the  number  of 
potential  participants  to  49.  After  approval  from  the  university’s  IRB,  persons  who  met  the  inclusion  criteria  were 
emailed  requesting  participation.  Participants  (n=16)  had  an  average  of  23  years  of  professional  instructional  design 
experience.  There  were  six  female  and  ten  male  participants. 

Of  the  16  participants,  7 (44%)  were  interviewed  at  AECT  while  9 (56%)  were  interviewed  in  the 
following  months  via  videoconference  and  phone  interviews.  In  addition,  those  who  agreed  to  participate  were  asked 
to  complete  a demographic  survey  that  allowed  the  research  team  to  determine  if  they  met  the  minimum  level  of 
expertise  required.  Criteria  included  10  years  of  professional  instructional  design  experience  (Ertmer  et  al.,  2008; 
LeMaistre,  1998;  Perez,  Johnson,  & Emery,  1995),  high  educational  degree  (Day  & Lord,  1992;  Eseryel,  2006; 
Korth,  1997),  and  currently  practicing/managing  instructional  design  (Ericsson  & Charness,  1994;  Rowland.  1992). 
All  16  participants  met  the  aforementioned  minimum  criteria  for  expertise.  The  research  team  believed  that  16 
interviews  would  be  enough  to  answer  the  research  question,  but  left  open  the  option  that  if  after  the  interviews  they 
felt  they  needed  more  data,  more  participants  could  be  found  and  more  interviews  conducted. 

Role  of  the  Researchers 

The  first  author  took  primary  responsibility  for  the  research,  including  designing  and  implementing  the 
study.  A faculty  member  and  a visiting  scholar  at  the  university  acted  as  consultants.  The  first  author  is  a doctoral 
student  in  an  educational  technology  program  at  a large  mid-western  university.  With  each  participant  she 
established  her  own  background  in  instructional  design.  This  was  done  to  foster  their  freedom  to  use  instructional 
design  terminology  during  the  interview.  However,  it  also  led  some  of  them  to  take  on  a “teacher”  perspective  since 
she  was  considered  a “novice”  or  “student.”  Whether  this  impacted  their  responses  during  the  interview  or  not, 
cannot  be  said. 
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Interview  protocol 

A six-page  data  collection  protocol  was  created  and  piloted  by  the  research  team.  It  included  the  steps  the 
researchers  should  complete  before,  during,  and  after  the  interview,  as  well  as  demographic  survey  questions, 
research  questions,  possible  interviewee  questions  and  appropriate  responses,  follow-up  questions,  and  an  interview 
check-out  list  (to  ensure  necessary  data  were  collected).  Jonassen  and  Hernandez-Serrano  (2002)  stated  that 
experienced  practitioner  stories  can  provide  relevant  information  about  strategies  they  use  in  the  instructional  design 
problem-solving  process.  They  listed  specific  steps  for  working  with  practitioners  to  identify  lessons  learned.  Some 
of  these  ideas  were  used  in  the  interview  protocol  as  probing  questions.  For  example,  participants  were  asked  to 
identify  the  problem  goals  and  to  describe  the  context  in  which  the  problem  occurred.  In  addition,  they  were  asked 
to  describe  the  solution  that  was  chosen  and  the  outcome  of  that  solution.  Participants  were  also  asked  to  identify 
any  lessons  learned  from  the  project. 

Prior  to  the  start  of  the  study,  the  research  team  conducted  three  practice  interviews  with  professors  in  the 
Educational  Technology  program  at  the  university  in  order  for  the  lead  researcher  to  familiarize  herself  with  asking 
the  interview  protocol  questions.  Subsequently,  the  research  team  refined  the  interview  protocol;  some  interview 
questions  were  added,  eliminated,  and  altered  for  better  interview  communication  procedures. 

Demographic  survey 

A demographic  survey  was  provided  via  email  to  participants  prior  to  interviews.  It  requested  the  following 
information:  participants’  gender,  age,  current  position,  number  of  years  in  that  position,  current  organization  or 
institution,  highest  degree  earned,  area  of  specialization,  number  of  years  in  the  instructional  design  field,  context 
and  years  of  instructional  design  positions,  and  instructional  delivery  formats  used. 

Data  Collection 

After  approval  from  the  university’s  IRB,  persons  who  met  the  inclusion  criteria  were  emailed  requesting 
participation.  Once  participation  was  accepted,  the  demographic  survey  and  consent  form  were  emailed  to 
participants.  Participants  returned  the  survey  and  consent  form  via  US  Postal  Service,  Email,  in-person,  or  fax. 

Interviews  were  conducted  over  a four-month  period.  Interviews  took  place  in  a face-to-face  manner  at 
AECT,  as  well  as  via  telephone  and  video-conferences;  at  a time  and  location  convenient  to  participants.  An  email 
was  sent  to  participants  verifying  interview  location  and  time  as  well  as  providing  them  a brief  question  to  prepare 
them  for  the  interview.  A semi-structured  interview  was  used  to  elicit  a story  about  a complex  or  challenging 
instructional  design  problem  on  which  the  instructional  designer  has  worked.  The  lead  researcher  began  the 
interview  by  stating: 

I would  like  you  to  start  by  telling  me  a story  about  an  instructional  design  project  you  worked  on  in  which 
you  had  to  solve  a complex  or  challenging  instructional  design  problem.  If  you  could,  I would  like  you  to 
start  with  you  entering  the  situation  and  continue  the  story  until  you  solved  the  problem.  Include  as  much  of 
the  following  as  is  needed  to  make  the  story  understandable:  setting,  context,  environment;  characters 
involved,  roles;  time  span  or  timeline;  and  actions,  events,  or  decisions.  I am  interested  in  hearing  your 
story  about  solving  the  instructional  design  problem.  Remember  the  ultimate  goal  is  to  identify  some  of  the 
rules  of  thumb  you  use  in  your  practice. 

Narratives  were  elicited  via  interview  questions  such  as:  (a)  What  major  constraints  or  functional  requirements 
existed  for  the  completion  of  the  task?  (b)  How  did  you  go  about  determining  possible  solutions?  (c)  What  was  the 
outcome  of  that  solution?  and  (d)  Were  there  any  lessons  from  this  ID  problem  you’ve  continued  to  utilize  in  solving 
future  ID  problems?  Probing  questions  were  used  by  the  lead  researcher  to  elicit  lessons  learned  and  extract  rules  of 
thumb  by  which  the  participant  operated  in  the  design  process  (e.g..  In  hindsight,  were  there  any  aspects  of  the 
project  that  could  have  been  improved?  What  types  of  lessons  did  you  take  away  from  this  ID  problem?  After 
having  these  experiences,  what  advice  would  you  give  to  someone  who  has  just  discovered  that  he  or  she  is  in  the 
same  situation?  What  is  your  greatest  challenge  when  trying  to  solve  ID  problems?).  Each  interview  lasted  between 
30  and  90  minutes.  All  interviews  were  audio/video-taped  and  transcribed  verbatim. 

Transcription  of  the  interview  data  was  completed  with  outside  assistance.  The  lead  researcher  verified  the 
accuracy  of  each  transcript  by  reading  through  the  transcript  while  listening  and  viewing  the  audio  and  videotaped 
interviews.  Corrections  to  transcripts  were  made  during  the  verification  process. 

Data  Analysis 

Phenomenological  methods  employed  by  Colaizzi  (1978)  and  modified  by  Moustakas  (1994)  were  used  to 
analyze  the  data.  Giorgi  (1985)  outlined  two  descriptive  levels  of  the  approach:  Level  one  utilizes  the  original  data 
(the  verbatim  transcript  of  the  interview)  and  can  be  considered  ‘naive’  descriptions,  whereas  level  two  contains  the 
researcher’s  interpretation  of  the  data,  where  through  reflective  analysis,  he/she  describes  the  structures  of  the 
experience.  These  structures  are  the  “underlying  structures  that  account  for  an  experience  being  what  it  is” 
(Moustakas,  1994,  p.  137)  and  are  the  ‘essence’  of  the  experience.  In  the  final  step,  the  researchers  combine  the  two 
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descriptions  (raw  data  and  interpreted  meanings)  into  a comprehensive  analysis  of  the  phenomenon.  For  example,  a 
level  one  comment  was,  “I  let  people  describe  the  problem.  And  again,  even  though  it  can  be  difficult  for  me,  I will 
not  ask  any  questions,  what  so  ever,  until  they  are  completely  finished.”  This  statement  is  verbatim  from  the 
interview.  The  level  two  analysis  led  to,  “the  instructional  designer  needs  to  really  listen  to  the  client  when  the  client 
is  describing  the  problem.”  From  these  two  descriptions,  the  rule  of  thumb  that  emerged  was,  “As  a designer  you 
need  to  listen  more  than  you  talk.” 

To  begin,  video  and  audio  transcripts  were  read  a number  of  times  in  order  to  obtain  an  overall  feeling  for 
the  interviews.  Significant  statements  related  to  instructional  design  heuristics  were  identified  in  the  transcripts. 
These  were  highlighted  and  written  on  index  cards  for  easier  organization.  Systematic  steps  (i.e.,  open  coding,  axial 
coding,  and  selective  coding)  were  used  to  generate  categories  of  heuristics  (Creswell,  2007).  Statements  were  then 
grouped  into  themes  (within  the  categories)  common  to  all  participants'  transcripts.  A theme  comprised  a recurring 
idea,  thread,  or  meaning  that  might  not  be  mutually  exclusive;  statements  sometimes  fit  into  more  than  one  theme 
(Graneheim  & Lundman,  2004).  Themes  were  then  described  as  an  in-depth  look  at  the  phenomenon.  Findings  were 
validated  with  the  participants  and  any  discrepancies  were  negotiated  or  noted  (Creswell,  2007). 

Trustworthiness  of  the  Study 

Trustworthiness  of  the  study  appears  through  the  systematic,  rigorous  steps  executed  in  the  data  collection 
and  analysis  procedures  (Patton,  2002).  The  development  of  an  interview  protocol  and  use  of  a pilot  study  ensured 
consistency  of  the  interviews.  The  interview  process  allowed  for  the  collection  of  rich,  detailed,  personal  narratives 
from  the  participants.  Reliability  of  the  data  was  enhanced  by  viewing  the  videotapes  multiple  times,  listening  to  the 
audiotapes  multiple  times,  and  verification  of  the  transcripts  (Patton,  2002). 

Results 

This  study  examined  the  heuristics  used  by  experienced  instructional  designers  when  solving  complex 
design  problems.  Analyses  of  interview  transcripts  led  to  the  identification  of  eight  categories  of  heuristics. 
Categories  included:  (1)  communication,  (2)  management,  (3)  learner/audience,  (4)  solutions/deliverables/outcomes, 
(5)  design  process,  (6)  design  team,  (7 ) design  problem,  and  (8)  client.  Within  each  category,  multiple  heuristics 
were  identified.  In  the  next  section,  we  describe  the  eight  categories,  the  heuristics  included  in  each  category,  and 
supporting  statements  made  during  the  interviews  by  participants.  It  should  be  noted  that  these  are  not  listed  in  any 
order  of  importance.  In  addition,  the  number  of  participants  who  referred  to  a specific  heuristic  is  listed  after  the 
heuristic. 

Communication 

The  communication  category  (see  Table  1)  includes  heuristics  that  relate  to  communication  between  the 
instructional  designer  and  his/her  client.  In  this  category  six  participants  discussed  listening  to  what  the  client  was 
saying  in  order  to  get  as  much  information  as  possible.  For  example,  “Go  to  the  meeting  and  keep  your  ears  open. 
They  are  going  to  say  things  there  that  you  are  going  to  need  to  know  about  later”  [E.B.]. 

Once  the  instructional  designer  felt  he/she  had  the  information  from  the  client,  five  participants  said  it  was 
necessary  to  verify  that  he/she  heard  that  information  correctly.  For  example,  “The  first  thing  I want  to  do  is  I verify 
things  that  I’ve  heard.  lust  literally,  honestly,  and  completely  verify  the  things  that  I’ve  heard.  Not  to  ask  a leading 
question,  but  just  simply  for  me  to  verify  that  what  I think  I’ve  heard  is  what  the  client  is  trying  to  say”  [J.Q.].  “I  put 
things  in  writing  and  I say  ‘do  you  agree  with  this?  Is  this  accurate?  Let’s  change  this  until  it’s  accurate’”  [S.P.].  The 
confirmation  message,  written  after  meetings  and  sent  usually  by  email,  seemed  to  be  an  additional  way  of 
documenting  and  verifying  statements  regarding  the  instructional  design  problem.  Seven  participants  discussed  the 
importance  of  visuals  and  documents  when  communicating  with  the  client.  That  documentation  was  very  important 
in  facilitating  communication.  As  one  participant  described,  “It  is  always  more  successful  for  me  to  show  people 
what  I am  thinking,  than  it  is  to  tell  them  about  it”  [E.B.].  Another  said,  “Visually  portray  whatever  process  model 
you  will  be  using  and  consider  sharing  that  with  the  client  or  the  primary  stakeholder  group  early  on  in  the  process 
to  help  make  sure  the  lines  of  communication  are  clear”  [R.B.]. 

In  addition,  three  participants  discussed  using  the  correct  lexicon  when  conversing  with  the  client  in  order 
to  facilitate  communication  and  reduce  misunderstandings.  “I  think  the  important  thing  for  any  designer  to  do  would 
be  to  provide  yourself  with  ample  time  to  learn  the  lexicon  ahead  of  time  so  that  you  can  more  easily  just  follow  on 
the  conversations  faster”  [P.L.].  This  means  the  instructional  designer  needs  to  understand  the  terminology  used  by 
the  client  and  only  use  instructional  design  terminology  if  it  is  necessary  to  the  conversation.  Otherwise,  it  was 
suggested  that  the  instructional  designer  use  terms  the  client  would  understand.  As  J.Q.  explained,  “We  use  our 
technical  terms,  but  when  we  talk  to  clients,  not  only  should  we  not  do  it  unless  we  explain  it,  I don’t  think  we 
should  do  it  unless  we  need  to  do  it.  It  builds  up  a barrier  in  a sense,  or  it  can,  with  a client  if  they  aren't  familiar 
with  this  terminology.” 


499 


Table  1 

Communication  Instructional  Design  Heuristics 

Heuristic No.  Participants 

As  a designer  you  need  to  listen  more  than  you  talk.  6/16 

Verify  all  information  received  from  the  client  to  prevent  miscommunication.  Send  a follow  5/16 

up  email  (confirmation  message)  after  meetings. 

Use  visuals  and  documents  to  prevent  miscommunication.  7/16 

The  instructional  designer  and  client  need  to  understand  the  lexicon  used  by  the  other.  3/16 


Management 

Managing  the  first  meeting  with  the  client  was  discussed  as  important  by  three  participants  (see  Table  2). 
One  participant  explained  how  the  first  meeting  progresses:  “I  would  say  to  [the  client],  ‘In  the  first  meeting,  you  are 
going  to  be  doing  a lot  of  the  showing  and  talking  because  I want  to  hear  all  your  perspectives  on  this  situation  that 
you  are  faced  with.  I will  be  taking  notes  and  if  you  have  any  materials,  etc.  send  them  to  me  ahead  of  time.’  And 
then  I will  say,  ‘I  will  either  give  you  some  initial  thoughts  on  the  situation,  or  if  the  problem  seems  quite  complex,  I 
will  take  it  away  and  come  back  in  a week  and  give  you  my  thoughts  on  where  we  should  go  from  then  on'”  [J.Q.]. 
This  indicates  that  the  instructional  designer  took  time  to  process  the  provided  information  and  did  not  just  jump 
into  designing  in  the  first  meeting  with  the  client.  R.B.  supported  this  notion  when  he  said,  “Don't  even  do  any 
instructional  design  things  in  the  first  meeting.” 

In  addition  to  managing  the  first  meeting  with  the  client,  five  participants  stated  how  the  instructional 
designer  needs  to  identify  the  constraints  placed  on  a project  as  well  as  negotiate  the  scope  of  the  project  with  the 
client.  “The  first  thing  I do  is  I look  for  the  constraints  that  have  been  placed  on  the  project.  Those  constraints  are 
the  key  to  design”  [A.G.].  M.G.  stated,  “It  was  really  ingrained,  very  early  in  my  head,  about  how  limitations  at  the 
end  impact  what  you  are  able  to  do  at  the  beginning,  and  so  understanding  what  those  limitations  are  really  makes  it 
more  successful  for  you  to  not  have  to  redo  work.”  When  looking  at  the  beginning  of  a design  project  and  the 
constraints,  eight  participants  discussed  how  a statement  of  work  should  be  created  and  verified  by  the  client.  As 
M.Sp.  noted,  “Getting  the  scope  of  the  effort  right  is  important.  And  I think  that  people  are  constantly 
underestimating  what  the  scale  of  the  training  design  effort  was  going  to  be,  they  just  never  scoped  it  correctly.  They 
underestimated  how  long  it  would  take,  how  much  and  what  kind  of  resources  would  be  required.”  Because  of  the 
constraints  placed  on  a project  by  the  client,  as  well  as  the  resources  provided,  four  participants  discussed  not  being 
able  to  create  “perfect”  instructional  design.  As  J.Q.  explained,  “One  cannot  often  do  the  best  project  that  you  would 
like  to.  You  can  do  the  best  instructional  design  with  the  resources  that  your  client  provides.” 
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Table  2 

Management  Instructional  Design  Heuristics 

Heuristic No.  Participants 

Don't  do  any  instructional  design  in  the  first  meeting.  3/16 

Look  for  constraints  that  have  been  placed  on  a project.  5/16 

Negotiate  the  scope  of  the  project.  Create  a statement  of  work  upfront.  8/16 

Do  the  best  ID  you  can  with  the  given  resources  and  constraints. 4/16 


Design  Team 

This  category  resulted  from  four  participants  describing  how  important  it  was  to  have  the  right  people  on 
the  team  (see  Table  3).  “Design  is  a people  process.  If  we  don't  have  tools,  if  we  don't  train  ourselves  to  think  about 
how  we  work  with  people  and  how  we  respect  people,  especially  including  our  coworkers,  that  has  a huge  impact  on 
what  we  are  a able  to  do  and  how  things  get  done”  [E.B.]. 

Some  participants  discussed  involving  everyone  from  the  beginning,  while  others  stated  that  they  only 
involve  certain  people  at  certain  times.  As  E.B.  stated,  “One  of  the  things  they  did  that  really  impressed  me  was,  we 
didn't  just  sit  with  one  writer,  he  called  in  the  whole  team.  The  entire  group  came.  We  didn’t  even  know  why  some 
of  them  might  be  necessary. . . but  that  rule  of  thumb  goes  something  like  this:  Get  together  all  the  excited,  exciting 
people  you  can  find  and  when  the  problem  is  truly  something  unique  to  them  bring  in  more,  not  less  and  don’t  ask 
yourself,  ‘well,  are  we  really  sure  we  need  one  of  these  or  one  of  those?’”  Whereas  W.C.  noted  that  more  than  three 
people  can  stifle  the  process,  “One  person  is  almost  never  as  smart  as  two.  Two  people  are  never  as  smart  as  three, 
but  when  you  begin  to  get  above  that,  it  gets  really  interesting.  You'll  get  lots  of  ideas,  but  you'll  have  a hard  time 
getting  finished.”  Those  who  discussed  this  category  agreed  that  instructional  design  is  done  in  teams  and  that  you 
have  to  work  well  with  other  people  in  order  to  come  to  a viable  solution.  “I  would  say  that  one  of  the  common 
problems  that  you  see  is,  especially  if  you’re  working  with  a team,  that  it’s  hard  to  get  internal  consistency.  It’s  hard 
to  get  everybody  to  really  communicate  enough  so  that  you  finally  come  to  an  agreement  on  what  it  is  that  you’re 
doing.  That’s  the  real  problem  ...”  [S.P.]. 

Table  3 

Design  Team  Instructional  Design  Heuristics 

Heuristic No.  Participants 

Design  is  a people  process.  4/16 

The  team  is  critical.  Involve  the  right  people  at  the  right  time. 4/16 


Design  Process 

The  design  process  category  (see  Table  4)  included  items  related  to  the  overall  instructional  design  process 
such  as  knowing  the  foundational  theories,  design  models,  and  past  solutions.  Six  participants  mentioned  that 
analysis  was  a very  important  step  in  the  instructional  design  process.  As  N.D.  stated,  “Analysis,  analysis,  analysis; 
you  can  never  do  enough  analysis.”  M.G.  also  stated,  “Making  sure  that  as  you  move  through  the  instructional 
design  process  that  you  come  back  to  the  analysis  component  again  and  again.” 

In  addition,  understanding  the  fundamentals  of  learning  theories,  instructional  design  models,  and 
principles  of  instruction  seemed  a predominant  theme  discussed  by  six  participants  throughout  the  interviews.  For 
example,  M.T.  said,  “You  need  to  know  the  theories.  You  need  to  know  the  models.  You  need  to  have  that 
foundation.  If  you  don't  have  that  foundation. . .that  almost  needs  to  be  your  ground  floor.”  D.M.  noted,  “We  need 
to  implement  those  principles  of  instruction  that  work.” 

Another  concept  discussed  by  five  participants  in  the  design  process  category  was  the  idea  of  using  ideas 
that  have  worked  in  the  past.  For  example,  W.C.  stated,  “Don't  try  to  break  too  much  new  ground.”  A.G.  expanded 
on  this  by  explaining  that  you  can  use  past  situations  that  have  worked,  but  to  add  to  them,  “Be  prepared  to  have  to 
think  abstractly,  and  don’t  just  copy  old  solutions,  try  to  add  something  new  in  every  design.” 
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Table  4 

Design  Process  Instructional  Design  Heuristics 

Heuristic No.  Participants 

During  the  ID  process  revisit  the  analysis  component  again  and  again.  6/16 

You  need  to  know  the  theories  and  models.  Implement  those  principles  of  instruction  that  6/16 

work. 

When  faced  with  something  complex,  draw  from  past  experiences.  5/16 


Learner/Audience 

The  learner/audience  category  (see  Table  5)  was  mainly  comprised  of  knowing  your  target  audience  and 
what  it  was  you  wanted  them  to  be  able  to  do  after  the  instructional  experience.  This  specific  heuristic  had  the 
largest  number  of  participants  (n=8)  discuss  it.  As  W.C.  stated,  “Invest  as  much  time  as  you  can  in  your  audience 
analysis  because  it  pays  probably  the  biggest  returns  of  all  of  them.  Task  analysis  gets  a lot  of  attention  because  it  is 
core  to  it,  but  a lot  of  times  people  don't  spend  enough  time  looking  at  learning  skills  and  other  things  about  the 
audience  and  if  you're  not  careful  you'll  miss  some  crucial  property  of  that  audience  in  terms  of  their  experience.” 

In  regards  to  the  performance  outcomes  of  the  learners,  seven  participants  discussed  knowing  what  it  was 
you  wanted  your  learners  to  be  able  to  do  at  the  end  of  the  instructional  experience.  “When  I approach  any 
instructional  design  problem,  I always  try  to  use  the  mantra  of,  ‘what  do  we  want  people  to  be  able  to  do?’  How  can 
you  create  an  instructional  activity  that  approximates  the  kinds  of  activities  they  are  going  to  be  doing  in  the  job  you 
are  trying  to  prepare  them  for?”  [P.P.].  B.B.  stated,  “Let’s  spend  our  time  giving  [learners]  opportunities  where  we 
show  them  what  success  looks  like.  Give  them  an  opportunity  to  practice  it.  And  give  them  feedback  about  how  to 
improve.  Teach  them  how  to  do  it  well,  give  them  practices  to  show  they  can  do  it  well  and  they’re  going  to  be 
motivated.”  This  indicates  that  not  only  should  an  instructional  designer  look  at  the  performance  outcomes  at  the 
end  of  the  instructional  experience,  but  also  look  at  the  desired  performances  throughout  the  experience. 

Interestingly,  it  appears  that  not  all  of  the  designers  agreed  about  the  specific  implementation  of  the  rules. 
For  example,  in  the  Learner/ Audience  category,  although  four  participants  discussed  including  scaffolding  in  the 
instructional  experience,  as  shown  in  the  heuristic  “Consider  utilizing  scaffolding  in  your  instructional  experience,” 
S.P.  felt  it  was  necessary  to  hold  off  on  using  scaffolding  in  the  beginning  whereas  B.W.  thought  it  should  be  used  in 
the  beginning  and  then  tapered  off  at  the  end  of  the  instruction.  “Don't  scaffold  at  first.  Don't  direct.  In  the  first 
version  you  want  to  give  the  minimal  amount  of  direction  and  support  that  you  can  give  - because  you  really  want  to 
find  out  what  it  is  the  student  needs.  Where  things  break  down.  And  then  start  trying  some  of  those  things  out  on  a 
verbal  basis”  [S.P.].  “Use  relatively  high  scaffolding  and  support  early  with  moving  toward  more  independent  work 
as  they  get  their  bearings”  [B.W.].  These  different  viewpoints  both  take  into  consideration  using  scaffolding,  but  the 
amount  and  placement  vary  depending  on  the  instructional  situation. 

Table  5 

Learner/Audience  Instructional  Design  Heuristics 

Heuristic No.  Participants 

Know  your  learners/target  audience  and  their  pre-requisite  knowledge.  8/16 

Determine  the  criterion  for  successful  performance.  7/16 

Give  learners  the  tools  they  need  to  succeed.  Consider  using  scaffolding  in  the  instructional  4/16 

experience. 


Solutions/Deliverables/Outcomes 

The  solutions/deliverables/outcomes  category  (see  Table  6)  has  three  heuristics.  Four  participants  described 
the  first,  whether  instruction  is  the  answer  to  the  client's  problem.  “It’s  really  important  that  you  recognize  that 
education  and  training  will  only  get  you  so  far,  education  and  training  only  are  appropriate  viable  solutions  when 
there  is  a skill  and  knowledge  deficit”  [B.B.].  Or  as  J.Q.  stated,  “I  always  listen  quietly  and  I’m  always  asking 
myself,  ‘is  it  actually  instruction  that  is  needed  here  or  not’ .” 

The  second  idea  was  discussed  by  eight  participants.  This  rule  is  to  approach  the  design  problem  with  the 
end  in  mind.  This  entails  considering  what  the  deliverables,  learning,  and  performance  outcomes  are.  “When  it 
comes  to  instructional  design,  I identify  what  the  deliverables  are,  what  do  we  want  to  be  able  to  accomplish  at  the 
conclusion  of  the  project.  And  then  work  backwards  from  there”  [M.Si.].  This  heuristic  is  very  similar  to  one  in  the 
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Learner/ Audience/Pedagogy  category:  Determine  what  it  is  you  want  your  learners  to  perform  after  the  instructional 
experience.  However,  it  differs  slightly  in  that  this  heuristic  is  the  way  to  approach  the  problem,  not  just  a step  in 
the  process.  As  B.B.  stated,  “Always  start  with  figuring  out  what  are  the  ways  you  want  people  to  change  as  a result 
from  this  experience.” 

In  addition,  four  participants  suggested  generating  and  providing  the  client  with  multiple  solutions  to 
solving  the  problem  and  letting  the  client  choose  which  one  to  implement.  For  example,  “We  got  to  the  point  where, 
after  developing  instructional  strategies,  we  developed  alternatives  for  several  instructional  strategies  so  that  even 
the  clients  could  make  some  judgments  about  which  might  be  best  or  which  might  be  preferred  given  the 
circumstances  of  that  specific  situation"  [R.B.],  “You  have  to  first  understand  a variety  of  solutions,  so  you  have  to 
have  an  idea  of  what  are  all  of  my  possibilities  of  outcome”  [P.L.]. 

Table  6 

Solutions/Deliverables/Outcomes  Instructional  Design  Heuristics 

Heuristic No.  Participants 

Ask  yourself,  “Is  instruction  the  solution  to  this  problem?”  4/16 

Approach  the  design  problem  with  the  end  in  mind.  What  are  the  deliverables?  What  are  the  8/16 

learning/performance  outcomes? 

Generate  multiple  possible  solutions  that  will  solve  the  problem.  4/16 


Design  Problem 

The  design  problem  category  (see  Table  7)  comprised  understanding  the  particular  unique  design  problem. 
This  category  had  the  fewest  participants  discussing  the  same  concept,  but  the  researchers  felt  these  heuristics  were 
important  enough  to  include.  Two  participants  mentioned  that  although  the  instructional  designer  might  have 
worked  on  similar  problems,  every  situation  is  unique.  As  B.B.  noted,  “I  think  because  every  single  situation  is 
different,  because  every  single  situation  is  unique,  that  taking  clients  language  to  figure  out  how  can  I turn  this  into  a 
viable  solution  that  actually  addresses  the  need  and  I am  not  just  doing  something  to  do  it  and  I’m  not  just  doing 
something  they  asked  to  do.”  E.B.  said,  “You  find  out  immediately  there  are  no  cut  and  dried  situations,  there  are  no 
situations  where  you  can  look  up  in  your  book  [acts  as  if  following  a diagram]  right. . .this  is  what  we  need  to  do.” 

In  addition,  three  participants  said  that  getting  to  the  core  of  the  problem  and  identifying  all  of  the  sub 
problems  was  important.  For  example,  “I  came  back  wondering  - have  I got  the  real  problem?  Have  they  just  sort  of, 
for  whatever  reason,  just  sort  of  given  me  pieces  of  the  problem  because  of  things  that  might  be  going  on  in  their 
own  organization  or  for  their  own  needs,  or  am  I forgetting  to  ask  something  about  the  problem  that  is  really 
important,  but  I didn’t  think  to  ask?”  [J.Q.].  A.G.  stated,  “I  think  that  every  design  problem  that  comes  to  us  has 
many  sub  problems  that  need  to  be  solved.  If  we  can  look  at  our  design  in  terms  of,  what  I call  layers,  of  the  design, 
we  find  that  we  solve  a bunch  of  individual  problems  which  then  integrate  together  into  one  coherent  design.” 

Table  7 

Design  Problem  Instructional  Design  Heuristics 

Heuristic No.  Participants 

Never  look  at  the  problem  at  face  value.  Get  to  the  core  of  the  problem  and  solve  all  the  sub-  3/16 

problems. 

Understand  that  every  design  situation  is  unique.  2/16 


Client 

In  the  client  category  (see  Table  8)  six  participants  discussed  determining  if  the  client  was  able  to  describe 
what  it  was  they  really  wanted  from  the  instructional  designer.  Once  this  was  communicated,  the  instructional 
designer  needed  to  “manage”  the  client's  expectations;  part  of  this  overlaps  with  developing  a statement  or  scope  of 
work.  As  N.D.  stated,  “Manage  the  client's  expectation.  So  don't  just  take  the  project  as  is.  Look  at  the  project,  look 
at  what  they  want  you  to  develop,  look  at  your  time  limits.”  As  M.Si.  said,  “Many  times  the  client  comes  to  you  and 
they’re  not  really  sure  what  they  want.  And  so  you  run  the  risk  of  proposing  something  that  will  satisfy  their  initial 
need  for  some  help,  some  assistance,  but  without  really  solving  the  instructional  design  situation  that  brought  them 
to  you  in  the  first  place.”  In  order  to  determine  what  the  client  wants,  the  instructional  designer  needs  to  listen.  As, 
P.L.  stated,  you  “understand  the  client  by  listening  to  them.  By  listening  to  what  questions  they  ask  you.” 

Another  theme  that  emerged  from  the  client  category  was  that  different  clients  have  different  levels  of 
cultural  sensitivity.  Five  participants  discussed  cultural  and  contextual  sensitivity  when  working  with  a client.  “You 
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have  to  be  sensitive  to  the  context.  It’s  probably  the  biggest  thing  that  will  help  you  be  successful  or  not”  [P.L.].  By 
creating  a trusting  relationship  utilizing  honesty,  the  instructional  designer  and  client  will  have  a more  successful 
partnership.  For  example,  a project  that  R.B.  discussed  was  centered  on  tax  accountants  who  worked  with  huge  sums 
of  money.  This  resulted  in  the  instructional  designers  maintaining  sensitivity  when  asking  about  “the  client’s 
companies,  what  were  their  sources  of  income  or  what  were  their  expenditures”  as  well  as  “how  questions  are 
perceived  by  the  client.”  Or  as  B.B.  stated,  “I  tell  them  why  I’m  doing  what  I’m  doing,  think  out  loud,  try  and  help 
them  understand  I’ve  got  their  best  interest  at  heart  and  go  from  there.  But  that  means  instructional  designers  really 
do  need  to  have  principles  and  practices  that  guide  them.”  The  trust  and  honesty  components  of  working  with  a 
client  go  hand  in  hand.  As  M.T.  phrased  it,  “Be  honest  with  the  client-  in  terms  of  your  recommendations.”  M.Sp. 


said,  “I  think  you  have  to  be  totally  brutally  honest  with  the  clients.” 

Table  8 

Client  Instructional  Design  Heuristics 

Heuristic No.  Participants 

Determine  if  the  client  knows  what  he/she  wants.  Manage  the  client’s  expectations.  6/16 

Be  sensitive  to  the  context  and  the  culture  of  the  client.  5/16 

Build  trust  with  clients  so  that  they  know  you  have  their  best  interests  at  heart.  3/16 

Be  honest  with  the  client.  3/16 


One  interesting  theme  from  the  results  was  how  many  times  participants  would  state  that  “Blank  is  the  key 
to  instructional  design.”  For  example:  A.G.  stated  that  constraints  are  the  key  to  design.  “The  first  thing  I do  is  I look 
for  the  constraints  that  have  been  placed  on  the  project.  Those  constraints  are  the  key  to  design.”  Whereas  M.T. 
thought  the  key  to  design  was  recruiting  the  right  people.  “..  .and  that  was  key-  recruiting  the  right  people.”  M.T. 
also  thought  that  asking  the  right  questions  of  the  client  as  well  as  looking  at  the  problem  from  different  perspectives 
was  also  key.  “Asking  the  right  questions  is  critical.  And  looking  at  it  from  many  different  perspectives.  Those  are 
really  key.”  B.B.  felt  that  understanding  what  performances  the  learner  should  do  at  the  end  of  the  instructional 
experience  was  key.  “If  I can  articulate  what  the  practice  experiences  should  be  and  what  the  critical  success  factors 
are  for  knowing  how  somebody's  performed  in  that  practice  what  success  looks  like  or  doesn't  look  like,  and  if  I can 
figure  out  what  demonstrations,  if  I can  get  that  in  place,  then  I’ve  done  what  I think  is  the  key  job  of  an 
instructional  designer.”  This  demonstrates  that  there  are  many  aspects  to  instructional  design  that  cannot  be  ignored. 
It  also  begs  the  question,  are  some  rules  of  thumb  more  important  to  the  instructional  design  process  than  others? 

Discussion 

Results  from  this  study  suggest  that  instructional  designers  use  a number  of  heuristics  in  their  ID  practice. 
Ertmer  et  al.  (2008)  also  described  the  use  of  rules  by  instructional  designers  in  their  study  examining  how  expert 
instructional  designers  used  knowledge  and  experience  in  the  instructional  design  process.  The  twenty-five 
heuristics  listed  above  were  found  as  themes  discussed  across  interviews.  The  list  of  heuristics  above  is  a partial  list 
due  to  the  limited  number  of  participants.  Some  of  these  heuristics  seem  to  be  common  sense,  such  as  verifying 
information  to  avoid  miscommunication.  Three  of  the  eight  categories  of  heuristics  (communication,  client,  and 
management)  would  appear  to  be  common  knowledge  to  business,  or  project  management  fields.  One  possible 
reason  for  the  overlap  would  be  the  common  use  of  problem  solving  in  the  different  fields.  In  the  management 
science  /organizational  research  (MS/OR)  field,  problem  solving  is  a common  component  (Evans,  1992).  Evans 
stated  that  many  MS/OR  problems  fall  into  the  ill-structured  category.  This  is  the  same  with  instructional  design 
(Jonassen  & Hernandez-Serrano,  2002).  Powell,  (1995)  described  three  heuristics  used  in  the  MS/OR  field  that  are 
similar  to  the  heuristics  found  in  this  study.  For  example,  one  of  the  heuristics  he  described  is  “decomposition”  of 
the  problem,  or  in  other  words,  breaking  down  the  problem  into  smaller  problems.  This  is  similar  to  the  ID  heuristic 
we  found:  "You  have  to  get  to  the  core  of  the  problem  and  solve  all  of  the  sub-problems.”  It  has  been  said  that 
breaking  down  the  problem  into  smaller  problems  allows  it  to  be  solved  easier  (Goel  & Pirolli,  1988;  Jonassen, 
1997).  In  Nelson's  (1988)  study,  experienced  instructional  designers  did  just  that  and  broke  down  the  larger  ill- 
structured  problem  in  order  to  work  on  sub-problems  separately. 

Another  MS/OR  heuristic  Powell  ( 1995)  noted  was  “Prototype:  Get  Something  Working”  (p.  1 17).  This  is 
similar  to  the  ID  heuristic  we  found:  “When  communicating  with  the  client  use  visuals  and  documents.”  Two  of  the 
participants  in  this  study  did  mention  using  prototypes  when  involved  in  instructional  design,  but  not  as  a rule  of 
thumb  they  follow.  More  in  common  with  this  ID  heuristic  is  Powell's  “Sketch  a graph:  Visualize”  heuristic  which 
is  a manner  of  representing  the  system  they  are  working  on  (p.  1 19).  Shanteau  (1992)  found  that  experts  use  decision 
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aids  or  visuals  when  working  on  a problem  that  requires  decision-making.  In  Hartley  and  Bruckmann’s  (2002)  book. 
Business  Communication,  they  discuss  how  to  use  visual  aids  as  a way  to  reduce  miscommunication.  These  all 
support  the  use  of  visuals  and  documents  when  communicating  with  a client. 

Other  heuristics  could  have  been  predicted  from  the  instructional  design  literature  and  the  fact  that  the 
majority  of  the  participants  are  academicians  in  the  field  of  instructional  design.  Those  would  consist  of  the  design 
process  heuristics,  the  learner/audience  heuristics,  and  the  solutions/deliverables/outcomes  heuristics,  as  many  of 
these  are  taught  in  introductory  instructional  design  courses.  For  example,  using  the  models  and  theories,  knowing 
your  target  audience,  and  determining  learning  outcomes  is  emphasized  in  a number  of  instructional  design 
textbooks  (e.g.,  Dick  & Carey,  1985;  Reiser  & Dempsey,  2002;  Romiszowski,  1981;  Smith  & Ragan,  1999).  Of  the 
instructional  design  heuristics  above,  we  have  yet  to  find  the  following  in  the  textbooks  or  journal  articles:  (a)  As  a 
designer  you  need  to  listen  more  than  you  talk,  (b)  Don't  do  any  instructional  design  in  the  first  meeting,  and  (c) 
Generate  multiple  possible  solutions  that  will  solve  the  problem.  This  last  heuristic  seems  to  follow  the  idea  that  ill- 
structured  problems  have  multiple  solutions  (Jonassen,  2000).  However,  the  idea  of  presenting  the  client  with 
alternatives  seemed  to  be  the  supporting  foundation  for  this  heuristic. 

What  is  unique  about  the  list  is  that  it  combines  these  different  areas  of  business,  project  management,  and 
design  into  one  overall  package  of  heuristics  that  instructional  designers  might  follow.  This  leads  us  to  ask  if  there 
should  be  a business/project  management  course  in  instructional  design  curricula. 

Not  every  participant  in  this  study  described  using  every  heuristic.  This  does  not  necessarily  mean  that  they 
do  not  use  that  heuristic  only  that  it  did  not  emerge  during  the  narrative  interview. 

Educational  or  scientific  importance  of  the  study 

Why  is  it  important  to  understand  the  heuristics  used  by  practicing  instructional  designers?  Because 
instructional  design  is  a problem-solving  process,  novice  instructional  designers  need  to  be  taught  more  about  what 
practicing  instructional  designers  actually  do,  rather  than  the  steps  in  a theoretical  instructional  design  model 
(Rowland.  1991;  Silber,  2007).  Allowing  novice  instructional  designers  to  learn  vicariously  through  the  stories  told 
by  practicing  instructional  designers  may  provide  them  with  experiences  they  would  not  encounter  otherwise 
(Jonassen  & Hernandez-Serrano,  2002).  If  we  can  determine  the  heuristics  practicing  instructional  designers  use,  the 
field  of  instructional  design  could  gain  an  additional  tool  to  use  in  the  education  of  novices.  Having  a list  of 
heuristics  may  help  make  designers  more  efficient  in  their  design  problem-solving  processes. 

The  underlying  rationale  for  examining  experts’  uses  of  heuristics  is  to  help  novice  instructional  designers 
develop  expertise.  However,  this  concept  needs  to  be  examined  in  future  research.  It  is  currently  not  yet  known  if 
these  heuristics  will  help  novices  develop  expertise.  The  results  of  this  study  suggest  that  in  addition  to  teaching 
novice  instructional  designers  the  foundational  models  and  theories,  we  should  also  be  including  business  and 
project  management  content. 

Limitations  and  Suggestions  for  Future  Research 

One  limitation  to  this  study  is  the  number  of  participants.  Sixteen  instructional  designers  took  part  in  this 
study.  A number  of  them  (14)  were  practicing  academicians  in  addition  to  being  instructional  designers.  Therefore, 
their  full-time  profession  is  not  that  of  an  instructional  designer.  They  were  chosen  because  of  their  experience  in  the 
field  as  well  as  their  willingness  to  be  interviewed  both  at  AECT  and  in  the  following  three  months.  The  fields  in 
which  these  16  instructional  designers  worked  did  not  represent  all  possible  areas  of  instructional  design;  for 
example,  designers  working  in  the  military  were  not  included.  This  is  because  the  researchers  did  not  have  access  to 
any  instructional  designers  currently  working  for  the  military.  In  order  to  create  a more  comprehensive  list  of 
heuristics  and  get  a better  sense  of  shared  heuristics  the  number  of  participants  would  need  to  be  increased  as  well  as 
the  fields  in  which  they  work. 

A second  limitation  was  the  different  modes  of  communication  for  interviews.  Interviews  were  conducted 
face-to-face,  via  video-conference,  and  by  telephone.  The  varying  modes  could  have  produced  different  results.  The 
face-to-face  interviews  might  have  provided  for  more  thoughtful  responses,  whereas  the  telephone  interviews  might 
reduce  interviewer  effects. 

Another  limitation  is  the  amount  of  access  the  researchers  had  to  the  participants.  Interviews  consisted  of 
only  one  30-90  minute  interview.  Follow-up  interviews  might  verify  the  use  of  heuristics  not  mentioned  in  the  initial 
interview.  Another  limitation  is  the  context  of  the  story  told  in  the  interviews.  Different  contexts  might  result  in 
different  heuristics  (e.g.,  military,  industry,  academia). 

Future  research  will  be  devoted  to  validating  these  heuristics  via  the  Delphi  technique  using  a panel  of 
carefully  selected  participants,  different  from  the  ones  for  this  phase.  The  Delphi  panel  will  consist  of  full-time 
practicing  instructional  designers.  In  addition,  the  panel  will  be  comprised  of  enough  participants  from  different 
fields  (e.g.,  military,  industry,  academia)  that  the  researchers  can  examine  if  there  is  a difference  in  heuristics  used  in 
the  different  fields.  In  addition,  the  researchers  will  focus  on  determining  the  best  methods  by  which  to  share  the 
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resulting  rules  of  thumb  with  novice  designers  and  whether  it  impacts  their  experiences  as  instructional  designers. 
Some  questions  we  would  like  to  ask  are:  1)  Can  we  teach  heuristics  to  novice  instructional  designers?  2)  What 
methods  should  we  use  to  provide  this  information  (stories,  cases,  guest  speakers)?  and  3)  How  does  this  impact 
their  practice? 

Conclusion 

“Procedures  for  intelligently  applying  past  knowledge  to  new  experience  often  seem  to  require  common 
sense  and  practical  rules  of  thumb  in  addition  to,  or  instead  of,  formal  analysis”  (Schank  & Abelson,  1977,  p.  3). 
This  study  elicited  rules  of  thumb  from  experienced  instructional  designers  as  a way  of  expanding  our  understanding 
of  what  practicing  instructional  designers  do  during  the  design  process.  This  knowledge  of  instructional  design 
heuristics  could  be  shared  with  novice  instructional  designers  while  they  are  learning  the  intricacies  of  becoming  an 
instructional  designer  possibly  as  a way  of  providing  them  some  wisdom  from  more  experienced  designers. 
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Appendix  A 

Participants’  Demographics  and  Description  of  Story  Contexts 
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